PL-TR-95-2156 


STATISTICAL  METHODOLOGY  AND 
ASSESSMENT  OF  SEISMIC  EVENT 
CHARACTERIZATION  CAPABILITY 


Mark  D.  Fisk 
Henry  L.  Gray 
Gary  D.  McCartor 


Mission  Research  Corporation 
735  State  Street 
P.O.  Drawer  719 
Santa  Barbara,  CA  93102 


31  October  1995 

Final  Report 

2  June  1993  -  2  September  1995 
Approved  for  public  release;  distribution  unlimited 


PHILLIPS  LABORATORY 
Directorate  of  Geophysics 
AIR  FORCE  MATERIEL  COMMAND 
HANSCOM  AFB,  MA  01731-3010 


SPONSORED  BY 

Advanced  Research  Projects  Agency  (DoD) 
Nuclear  Monitoring  Research  Office 
ARPA  ORDER  No.  A-128 

MONITORED  BY 
Phillips  Laboratory 
CONTRACT  No.  F19628-93-C-0117 


The  views  and  conclusions  contained  in  this  document  are  those  of  the 
authors  and  should  not  be  interpreted  as  representing  the  official  policies, 
either  express  or  implied,  of  the  Air  Force  or  the  U.S.  Government. 


This  technical  report  has  been  reviewed  and  is  approved  for  publication. 


(^L 


JAMES  F.  LEWKOWICZ 
<26ntract  Manager 


/t 


Sciences  Division 


XL. 


/ 

/- 

li 


/ 

(  r 


jAmesf.  l'ewkowicz 


.JMrector 

Earth  Sciences  Division 


This  report  has  been  reviewed  by  the  ESC  Public  Affairs  Office  (PA)  and  is 
releasable  to  the  National  Technical  Information  Service  (NTIS). 


Qualified  requestors  may  obtain  additional  copies  from  the  Defense  Technical 
Information  Center.  All  others  should  apply  to  the  National  Technical 
Information  Service. 

If  your  address  has  changed,  or  if  you  wish  to  be  removed  from  the  mailing 
list,  or  if  the  addressee  is  no  longer  employed  by  your  organization,  please 
notify  PL/IM,  29  Randolph  Road,  Hanscom  AFB,  MA  01731-3010.  This  will 
assist  us  in  maintaining  a  current  mailing  list. 

Do  not  return  copies  of  this  report  unless  contractual  obligations  or  notices  on 
a  specific  document  requires  that  it  be  returned. 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  NO.  0704.0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching 
existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this 
burden  estimate  or  any  other  aspect  of  the  collection  of  information.  Including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services, 
Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and 
Budget,  Paperwork  Reduction  Project  (0704-0188),  Washington.  DC  20S03. _ _ 


1 .  AGENCY  USE  ONLY  (Leave  Blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

3 1  Oct  1 995  Final  Report  2  Jun  1993  to  2  Sep  1995 


4.  TITLE  AND  SUBTITLE 

Statistical  Methodology  and  Assessment  of  Seismic 
Characterization  Capability 


6,  AUTHOR(s) 

Mark  D.  Fisk 

Henry  L.  Gray’"  Gary  D.  McCartor’" 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 
Mission  Research  Corporation 
735  State  Street,  P.O.  Drawer  719 
Santa  Barbara,  C A  93 1 02-07 1 9 


5.  FUNDING  NUMBERS 
PE62301E 

PR  NM93  TA  GM  WU  AL 
Contract  F 1 9628-93-C-Ol  17 


8.  PERFORMING  ORGANIZATION  REPORT  NUMBER 
MRC-R-1509 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 
Phillips  Laboratory 
29  Randolph  Road 

Hanscom  Air  Force  Base,  MA  01731-3010 
Contract  Manager:  James  Lewkowicz/GPEH 


11.  SUPPLEMENTARY  NOTES 


10.  SPONSORING/MONITORING  AGENCY 
REPORT  NUMBER 

PL-TR-95-2156 


Southern  Methodist  University,  Dallas,  TX 


12a.  DISTRIBUTION/AVAILABILITY  STATEMENT  12b.  DISTRIBUTION  CODE 

Approved  for  public  release;  distribution  unlimited 


13.  ABSTRACT  (Maximum  200  words) 

This  project  has  focused  on  developing  and  applying  statistical  methods  to  perform  seismic  event  characterization/identification  and 
on  quantifying  capabilities  with  regard  to  monitoring  of  a  Comprehensive  Test  Ban,  The  report  is  divided  into  four  separate  but 
related  sections.  First,  an  automated  procedure  is  described  to  categorize  seismic  events,  based  on  multivariate  analysis  of  features 
derived  from  seismic  waveforms.  The  procedure  is  intended  to  aid  CMR  subscribers  in  focussing  upon  events  of  interest.  Second, 
preliminary  event  identification  results  are  presented  for  a  seismic  event  which  occurred  on  5  January  1995  in  the  Southern  Ural 
Mountains  region.  This  was  a  shallow  event  which  occurred  in  an  area  of  low  seismicity  and  near  where  several  PNEs  had  been 
conducted.  Third,  various  statistics  are  compiled  regarding  1786  seismic  events  which  occurred  between  11  January  1995  and  12 
February  1995  and  were  detected  by  a  set  of  30  GSETT-3  Alpha  stations.  The  objective  of  this  study  is  to  determine  how  many 
events  can  be  screened  with  high  confidence  as  due  to  natural  seismicity,  based  on  teleseismic  measures  of  depth,  Ms:mb  and 
location  (offshore/onshore).  Of  the  remaining  events,  the  number  with  useful  regional  discriminants  is  quantified.  Last,  we  evaluate 
the  number  of  remaining  ambiguous  events  lacking  adequate  event  characterization  data  to  identify  them,  based  on  the  Alpha  network 
which  existed  at  the  time  of  this  study.  Fourth,  a  fundamental  problem  is  addressed  of  how  to  utilize  multivariate  discriminant  data 
from  a  multi-station  network  in  order  to  optimize  the  power  of  the  outlier  test  for  fixed  false  alarm  rate.  Analytic  expressions  for  the 
power  of  three  outlier  tests  are  derived  and  a  parametric  study  is  presented  in  order  to  determine  conditions  for  which  a  particular  test 
,  is  favored  over  the  others. 


14.  SUBJECT  TERMS 

Seismic  Event  Characterization 


Seismic  Event  Characterization  Comprehensive  Test  Ban 

Outlier  Analysis  Regional  Discriminants 

GSETT-3  Alpha  Network 

17.  Security  CLASSIFICATION  1 18.  Security  CLASSIFICATION  I  19.  Security  CLASSIFICATION 


OF  REPORT 
UNCLASSIFIED 

NSN  7540-01-280-5500 


OF  THIS  PAGE 
UNCLASSIFIED 


OF  ABSTRACT 
UNCLASSIFIED 


15.  NUMBER  OF  PAGES 

_ 90 _ 

16.  PRICE  CODE 


20.  LIMITATION 
OF  ABSTRACT 
SAR 

Standard  Form  298  (Rev.  2-89) 
Prescribed  by  ANSI  Std.  239-18 
298-102 


UNCLASSIFIED 

SECURITY  CLASSIFICATION  OF  THIS  PAGE 
CLASSIFIED  BY: 


Table  of  Contents 


Executive  Summary . ix 

Bibliography  of  Publications  Sponsored  by  Contract . xii 

1 .  A  Statistical  Procedure  for  Categorizing  Seismic  Events . 1 

1.1.  Introduction . 1 

1.2.  Overview  of  Technical  Approach . 3 

1.2.1.  Event  Screening  . 3 

1.2.2.  Population  Analysis  . 5 

1.3.  Definition  of  Event  Categories  . 7 

1.4.  Examples  of  Event  Categorization . 8 

1.4.1.  Categorization  Results  for  WMQ  Events  . 1 1 

1.4.2.  Categorization  Results  for  ARCESS  Events . 14 

1.4.3.  Categorization  Results  for  KNB  and  MNV  Events  . 16 

1.5.  Summary . 19 

2.  Preliminary  Identification  Analysis  of  the  950105  Urals  Event . 20 

2.1.  Introduction . 20 

2.2.  Data . 21 

2.2.1.  Amplitude  Ratios  Prior  to  Applying  Distance  Corrections  . 22 

2.2.2.  Amplitude  Ratios  After  Applying  Distance  Corrections . 24 

2.3.  Categorization  Results  for  the  950105  Event  . 28 

2.4.  Conclusions  and  Future  Plans . 28 

3.  Assessment  of  Current  Event  Characterization  Capability  using  GSETT-3  Alpha 

Network  Data . 30 

3.1.  Introduction . 30 

3.2.  GSETT-3  Data . 31 

3.2. 1 .  Events  Per  Alpha  Station  . 3 1 

3.2.2.  Events  Versus  Location  and  Location  Uncertainty  . 33 

3.2.3.  Events  Versus  Depth  and  Detected  Depth  Phases . 35 

3.2.4.  Events  Versus  Magnitude  . 37 

3.2.5.  Regional  High-Frequency  Amplitude  Ratios . 39 

3.3.  Utility  of  Event  Characterization  Parameters  . 51 

3.4.  Conclusions  and  Recommendations  . 55 


4.  Optimizing  Multivariate  Network  Evidence  for  Outlier  Detection . 57 

4.1.  Introduction . 57 

4.2.  Technical  Approach . 57 

4.3.  Parametric  Power  Comparison  . 59 

4.4.  Conclusions  and  Recommendations  . 70 


IV 


List  of  Figures 


1.  Schematic  of  first-cut  event  screening . 4 

2.  Relationship  of  Categories  3-10  to  the  cumulative  probability  of  the  log  likelihood  ratio 

when  the  null  hypothesis  is  true . 9 

3.  Locations  of  ARCESS,  CDSN  station  WMQ,  LNN  stations  KNB  and  MNV,  and  the 

seismic  events  used  in  this  study . 10 

4.  Locations  of  CDSN  station  WMQ,  16  nuclear  explosions  in  Kazakhstan,  the  880929 

nuclear  explosion  at  Lop  Nor  and  23  earthquakes  in  China  and  nearby  countries . 11 

5.  Distance-Corrected  Pn/Lg  in  nine  frequency  bands  for  23  earthquakes  and  17  nuclear 

explosions  recorded  by  WMQ . 12 

6.  Categorization  results  for  events  recorded  by  station  WMQ . 13 

7.  Outlier  test  results  for  events  recorded  by  station  WMQ . 13 

8.  Locations  of  the  ARCESS  array,  3  nuclear  explosions  on  Novaya  Zemlya,  5  Spitsbergen 

and  24  Steigen  earthquakes . 14 

9.  Distance-corrected  Pn/Sn  measurements  in  six  frequency  bands  for  events  recorded  by 

ARCESS . 15 

10.  Categorization  results  for  events  recorded  by  ARCESS . 15 

11.  Locations  of  LNN  stations  KNB  and  MNV  (among  others)  relative  to  NTS.  On  the  left 

is  an  enlarged  map  of  NTS  showing  epicenters  of  the  nuclear  explosions  and 
earthquakes . 16 

12.  Pn/Lg  measurements  in  the  6-8  Hz  band  versus  ML(coda)  for  events  recorded  by 

KNB . 17 

13.  Pn/Lg  measurements  in  the  6-8  Hz  band  versus  ML(coda)  for  events  recorded  by 

MNV . 18 

14.  Categorization  results  for  events  recorded  by  KNB  and  MNV . 18 

15.  Locations  of  seismic  stations,  arrays  and  events  used  in  our  analysis  of  the  950105 

Southern  Urals  event  (ORID=273228) . 21 

16.  Uncorrected  Pn/Lg  in  the  6-8  Hz  band  versus  distance  for  earthquakes,  nuclear 

explosions,  and  the  950105  Urals  event . 23 


17.  Uncorrected  Pn/Sn  in  the  6-8  Hz  band  versus  distance  for  earthquakes,  nuclear 
explosions,  and  the  950105  Urals  event . 23 


18.  Uncorrected  Pn/Sn  measurements  in  nine  frequency  bands  for  3  Novaya  Zemlya 
nuclear  explosions,  24  Steigen  earthquakes  and  the  950105  event  recorded  by 
ARCESS . 24 


19.  Uncorrected  Pn/Lg  measurements  in  eight  frequency  bands  for  the  950105  event 

recorded  by  ARU  and  OBN  and  earthquakes  and  nuclear  explosions  recorded  by 
WMQ . 25 

20.  Same  as  Figure  19,  but  for  uncorrected  Pn/Sn . 25 

21.  Distance-corrected  Pn/Sn  values  in  nine  frequency  bands  for  events  recorded  by 

ARCESS . 26 


22.  Distance-corrected  Pn/Lg  measurements  in  eight  frequency  bands  for  the  950105  event 
recorded  by  OBN  and  earthquakes  and  nuclear  explosions  recorded  by  WMQ . 27 


23.  Same  as  Figure  22,  but  for  distance-corrected  Pn/Sn . 27 

24.  Categorization  results  for  the  950105  event,  29  earthquakes  in  the  Steigen  and 

Spitsbergen  regions,  and  3  nuclear  explosions  at  the  Novaya  Zemlya  test  site . 29 

25.  Locations  of  30  Alpha  stations  and  1786  seismic  events  recorded  by  them  between  1 1 

January  and  12  February  1995 . 31 

26.  Number  of  overall  and  regional  events  per  Alpha  station . 33 

27.  Distribution  of  events  versus  90%  confidence  location  error  ellipse  area . 34 

28.  Scatter  plot  of  90%  confidence  location  error  ellipse  area  versus  mb . 34 

29.  Distribution  of  events  vs.  hypocentral  depth  estimate . 35 

30.  Distribution  of  events  vs.  number  of  depth  phases  per  event . 36 

31.  Cumulative  distribution  of  events,  with  or  without  one  or  more  detected  depth  phases, 

versus  the  upper  bound  of  the  95%  confidence  interval  for  depth . 36 


32.  Distributions  of  events  versus  mb,  Ms  and  Ml . 37 

33.  Scatter  plot  of  Ms  versus  mb . 38 

34.  Scatter  plot  of  Ml  versus  mb . 38 

35.  Number  of  regional  events  per  Alpha  station . 40 

36.  Distribution  of  regional  events  versus  the  number  of  stations  per  event . 41 

37.  Pn/Sn  in  4  frequency  bands  versus  Alpha  station.  Circles  indicate  SNR  <  2  for  either 

numerator  or  denominator,  while  asterisks  indicate  SNR  >  2  for  both  numerator  and 
denominator . 42 


38.  Same  as  Figure  37,  but  for  Pn/Lg. 


43 


i 


VI 


39.  Same  as  Figure  37,  but  for  Pg/Sn . 44 

40.  Same  as  Figure  37,  but  for  Pg/Lg . 45 

41.  Pn/Sn  in  4  frequency  bands  versus  distance.  Markers  are  defined  as  in  Figure  37.  ..46 

42.  Same  as  Figure  41,  but  for  Pn/Lg . 47 

43.  Seismic  waveforms  for  a  Gulf  of  California  event  which  was  recorded  by  the  Pinedale 

array.  Large  amplitude  noise  spikes  were  present  on  two  of  the  channels . 49 


44.  Sn  travel  times  versus  distance  (stars)  based  on  phase  picks  by  seismic  analysts  for  the 
one-month  data  set.  Also  plotted  are  the  IASPEI91  Sn  travel  time  curve  (solid  line)  and 
the  theoretical  velocity  window  (dashed  lines)  used  to  make  automated  Sn 


measurements . 50 

45.  Distribution  of  events  with  various  available  discriminants . 52 


46.  Locations  of  events  which  can  be  classified  as  deep,  offshore,  with  mb-M5  <  1.2,  or 

with  at  least  one  high-frequency  regional  amplitude  ratio.  Markers  are  coded  by 
category . 53 

47.  Locations  of  events  for  which  none  of  the  considered  teleseismic  and  regional 

discriminants  are  available . 54 


48.  Power  comparison  of  the  three  tests  for  0]  =02“  ^ . 

49.  Power  comparison  of  the  three  tests  for  0|  =  1  and  02  =  2 . 62 

50.  Power  of  Tests  1  (solid),  2  (dashed)  and  3  (dotted)  versus  p  for  0,  =  02  =  1  and  various 

values  of  Ap]  and  Ap2  from  0  to  3 . 63 

5 1 .  Same  as  Figure  50,  except  that  02  =  2 . 64 

52. Difference  in  the  power  of  Test  2  and  Test  1  versus  correlation  for  various  values  of  Api 

and  Ap2  from  0  to  4 . 65 

53.  Difference  in  the  power  of  Test  3  and  Test  1  versus  correlation  for  various  values  of 

Ap]  and  Ap2  from  0  to  4 . 66 

54.  Power  of  Tests  2  (solid),  4  (dashed)  and  5  (dotted)  versus  p  for  0]  =  1,  CJ2  =  2  and 

various  values  of  Apj  and  Ap2  from  0  to  3 . 68 

55.  Difference  in  the  power  of  Test  4  and  Test  2  versus  correlation  for  various  values  of 

Ap]  and  Ap2  from  0  to  4 . 69 


56.  Difference  in  the  power  of  Test  5  and  Test  2  versus  correlation  for  various  values  of 
Ap]  and  Ap2  from  0  to  4 . 70 


List  of  Tables 


1 .  Summary  of  Event  Category  Definitions . 8 

2.  Summary  of  seismic  data  sets  used  in  this  study . 10 

3.  Summary  of  seismic  data  sets  used  in  the  analysis  of  the  950105  Urals  Event . 22 

4.  Alpha  station  names,  operation  start  dates,  locations,  and  types . 32 

5.  Description  of  regional  phase  amplitude  measurements . 39 


viii 


Executive  Summary 


This  effort  has  focused  on  developing  and  applying  statistical  methods  to  perform  seismic  event 
characterization/identification  and  on  quantifying  identification  capabilities  with  regard  to  seismic 
monitoring  of  a  Comprehensive  Test  Ban  Treaty  (CTBT).  Difficulties  associated  with  seismic 
event  identification,  within  the  context  of  CTBT  verification,  include  the  fact  that  small  events  are 
seen  at  fewer  stations  or  arrays,  possibly,  at  only  a  single  station.  In  addition,  there  are  many 
thousands  of  competing  earthquakes,  mining  blasts  and  other  mining-induced  events  worldwide 
per  year.  Thus,  robust  automated  methods  are  needed  to  utilize  potentially  limited  information 
and  to  reduce  the  workload  of  human  analysts  to  a  manageable  level. 

While  useful  discriminants  such  as  Ms:mb  and  determination  of  focal  depth  can  be  measured  and 
provide  robust  discrimination,  typically  for  large  events  (mb>4.5)  seen  at  teleseismic  distances  by 
many  stations,  useful  measurements  of  these  discriminants  cannot  currently  be  obtained  for  small 
events  seen  only  at  regional  distances  by  a  small  number  of  stations.  Furthermore,  seismic  signals 
seen  at  regional  distances  from  small  sources  are  generally  quite  complicated  and  exhibit 
dramatic  dependence  on  regional  geophysics;  even  the  most  promising  regional  discriminants  can 
vary  widely  in  different  regions.  Hence,  region-specific  information  regarding  seismic 
discriminants  is  vital  in  distinguishing  nuclear  explosions  from  other  events.  Unfortunately, 
relevant  ground-truth  data,  particularly  for  small  underground  nuclear  explosions,  do  not  exist  for 
most  regions.  In  addition,  it  has  yet  to  be  shown  that  a  discrimination  rule,  established  in  a  region 
for  which  data  exist,  can  be  transported  effectively  to  a  new  region.  Thus,  for  many  cases,  seismic 
event  identification,  within  the  context  of  CTBT  or  NPT  (Non-Proliferation  Treaty)  monitoring,  is 
a  problem  of  identifying  unusual  events,  i.e.,  outliers. 

This  report  is  divided  into  four  separate  but  related  sections.  In  Section  1,  we  describe  a  technical 
approach  to  categorize  seismic  events  with  these  and  other  considerations  in  mind.  Our  objective 
is  to  provide  an  automated  procedure  to  categorize  events,  based  on  multivariate  analysis  of 
features  derived  from  seismic  waveforms,  and  to  make  this  information  available  in  a  database  at 
the  Center  for  Monitoring  Research  (CMR).  Subscribers  will  be  able  to  obtain  this  information,  to 
aid  in  focussing  upon  events  of  particular  interest. 

Since  there  are  only  a  small  number  of  mining  blasts  that  occur  annually  above  magnitude  3,  our 
primary  concern  for  monitoring  above  this  threshold  is  to  be  able  to  distinguish  nuclear 
explosions  from  earthquakes.  Thus,  we  focus  on  this  case  here.  Monitoring  below  this  threshold 
poses  greater  difficulty  and  will  be  addressed  in  a  subsequent  report  under  a  new  ARPA  contract. 


IX 


We  describe  an  approach  which  first  determines  those  events  that  are  of  natural  seismic  origin, 
based  on  confidence  intervals  for  depth,  location  (offshore  versus  onshore)  and  Ms:mb.  This 
analysis  applies  to  only  those  events  for  which  these  parameters  can  be  estimated,  currently  those 
observed  at  teleseismic  distances  by  many  stations.  Further  analysis  is  then  performed,  using  a 
multivariate  population  (i.e.,  outlier)  analysis  for  remaining  regional  events.  The  outlier  method 
has  sufficient  generality  to  utilize  any  combination  of  teleseismic  and  regional  discriminants  seen 
at  single  or  multiple  stations.  The  procedure  may  be  fully  automated  to  categorize  events  and  to 
test  all  appropriate  assumptions  to  ensure  validity  of  the  results.  Fisk  et  al.  (1993,  1994)  describe 
the  methodology  in  detail  and  provide  numerous  results  of  applications  to  seismic  data. 

Although  the  method  can  be  used  to  perform  a  hypothesis  test  with  a  controlled  false  alarm  rate 
(e.g.,  Fisk  et  al.,  1994),  our  focus  here  is  to  use  the  population  analysis  to  categorize  events  rather 
than  providing  a  “yes/no”  decision.  We  define  event  categories  based  on  the  discriminant  analysis 
described  below.  Categories  are  intentionally  defined  in  a  non-judgemental  manner,  but  with  a 
statistically  rigorous  interpretation.  We  present  examples  of  the  categorization  procedure  to 
regional  data  sets  consisting  of  earthquakes  and  underground  nuclear  explosions  recorded  by  the 
ARCESS  array,  CDSN  station  WMQ,  and  LNN  stations  KNB  and  MNV. 

In  Section  2,  we  present  preliminary  event  identification  results  for  a  seismic  event  which  occurred 
on  5  January  1995  in  the  Southern  Ural  Mountains  region.  This  was  a  shallow  event  which 
occurred  in  an  area  of  low  seisrmcity  and  near  where  several  PNEs  had  been  conducted.  Of  further 
interest  was  the  fact  that  its  relatively  small  value  of  Mgmb  was  considered  inconsistent  with 
identification  as  a  shallow  earthquake.  A  press  release  by  the  Moscow  ITAR-TASS  World  Service, 
6  January  1995,  reported  a  statement  by  the  Russian  Federation’s  State  Technical  Inspectorate  that 
this  event  was  a  tectonic  shock  at  Pit  Number  2  of  the  Silvinit  Joint-Stock  Company,  caused  by  an 
earthquake  with  its  epicenter  near  the  settlement  of  Chashkino  in  Perm  Oblast.  The  announcement 
also  indicates  that  the  tectonic  shock  led  to  the  destruction  of  ventilation  systems  at  the  mine  and 
to  considerable  emissions  of  explosive  gases  -  methane  and  hydrogen  -  which  subsequently  ignited 
and  exploded. 

The  objective  of  our  analysis  is  to  independently  assess  the  identification  of  this  event  by 
comparing  its  seismic  data,  recorded  by  the  ARCESS  and  KVAR  arrays  and  stations  ARU  and 
OBN,  to  that  of  previous  earthquakes  and  nuclear  explosions.  Since  the  950105  event  had  a 
magnitude  of  mb  4.35,  considerably  larger  than  would  be  produced  by  a  large  mining  blast,  we 
focus  here  on  characterizing  this  event  as  either  an  earthquake,  a  nuclear  explosion  or  neither. 
(Walter,  1995,  and  others  have  explicitly  considered  other  alternatives  such  as  a  mine  collapse.) 


In  Section  3,  we  compile  various  statistics  regarding  1786  seismic  events  which  occurred  between 
11  January  1995  and  12  February  1995  and  were  detected  by  a  set  of  30  Alpha  stations  as  part  of 
GSETT-3  (Group  of  Scientific  Experts  Technical  Test-3).  Seismic  data  from  these  stations  were 
transmitted  to  the  CMR  where  the  records  were  analyzed.  The  objective  of  this  study  is  to  assess 
the  numbers  and  characteristics  of  events  that  can  be  expected  to  be  observed  by  the  Alpha 
stations  during  a  given  period,  including  the  availability  and  utility  of  event  characterization 
parameters  that  can  be  used  to  identify  them.  High-frequency  regional  phase  amplitudes  and  Ms 
were  computed  for  the  events  during  this  period,  as  well  as  location,  depth  and  mb  parameters. 

We  examine  how  many  events  can  be  screened  with  high  confidence  as  due  to  natural  seismicity, 
based  on  teleseismic  measures  of  depth,  location  and  Ms^mb.  Of  the  remaining  events,  we 
examine  how  many  have  useful  regional  discriminants  such  that  they  could  potentially  be 
identified,  considering  primarily  high-frequency  regional  P/Sn  and  P/Lg  amplitude  ratios.  We  also 
assess  their  utility  with  respect  to  signal-to-noise  ratios.  Last,  we  evaluate  the  number  of 
remaining  ambiguous  events  lacking  adequate  event  characterization  data  to  identify  them,  based 
on  the  Alpha  network  which  existed  at  the  time  of  this  study. 

In  Section  4,  we  address  a  fundamental  problem  of  how  to  utilize  multivariate  discriminant  data 
from  a  multi-station  network  in  order  to  optimize  the  power  of  the  outlier  test  for  fixed  false  alarm 
rate.  In  general,  there  are  numerous  types  of  regional  measurements  that  can  be  used  as 
discriminants,  which  may  be  available  at  multiple  (e.g.,  2  to  5)  Alpha  stations  and  it  is  not  clear 
from  the  outset  how  to  best  combine  this  information  in  a  test  for  outliers.  We  present  three 
approaches  to  this  problem.  The  first  is  to  simply  insert  all  available  features  that  are  thought  to 
discriminate  into  the  likelihood  ratio.  The  second  is  to  perform  an  optimal  weighting  of  some  or 
all  of  the  discriminants  first  and  then  insert  the  weighted  combinations  into  the  likelihood  ratio. 
The  third  is  to  perform  separate  tests,  based  on  the  likelihood  ratio  and  subsets  of  discriminants, 
calling  an  event  an  outlier  if  it  is  found  to  be  an  outlier  by  any  of  the  individual  tests.  The 
significance  levels  of  the  individual  tests,  in  this  case,  must  be  appropriately  modified  to  maintain 
the  overall  false  alarm  rate  of  the  combined  results. 

We  derive  analytic  expressions  for  the  power  of  each  of  these  three  tests.  We  then  present  a 
parametric  study  in  which  the  power  of  the  three  tests  are  compared  in  order  to  determine 
conditions  for  which  a  particular  test  is  favored  over  the  others.  Last,  we  provide  some 
conclusions  and  recommendations  for  operational  application  to  seismic  monitoring. 
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1.  A  Statistical  Procedure  for  Categorizing  Seismic  Events 

1.1.  Introduction 

Objectives  for  an  international  CTBT  (Comprehensive  Test  Ban  Treaty)  monitoring  system, 
stated  in  the  United  States  Working  Paper  for  the  Committee  on  Disarmament  (May  18,  1994), 
are  detection  and  identification  of  nuclear  explosions  down  to  a  few  kilotons  or  less.  Difficulties 
associated  with  seismic  event  identification  down  to  this  level  include  the  fact  that  small  events 
are  seen  at  fewer  stations  or  arrays,  possibly,  at  only  a  single  station.  In  addition,  there  are  many 
thousands  of  competing  earthquakes,  mining  blasts  and  other  mining-induced  events  worldwide 
per  year  (Ringdal,  1984;  Lilwall  and  Douglas,  1985).  While  useful  discriminants  such  as  Ms-mb 
and  determination  of  focal  depth  can  be  measured  and  provide  robust  discrimination,  typically  for 
large  events  (mb>4.5)  seen  at  teleseismic  distances  by  many  stations  (e.g.,  Blandford  et  al.,  1992), 
useful  measurements  of  these  discriminants  cannot  be  obtained  currently  for  small  events  seen 
only  at  regional  distances  by  a  small  number  of  stations.  Thus,  robust  automated  methods  are 
needed  to  utilize  potentially  limited  information  and  to  reduce  the  workload  of  human  analysts  to 
a  manageable  level. 

Furthermore,  seismic  signals  seen  at  regional  distances  from  small  sources  are  generally  quite 
complicated  and  exhibit  dramatic  dependence  on  regional  geophysics;  even  the  most  promising 
regional  discriminants  can  vary  widely  in  different  regions  (Fisk  et  al.,  1994).  Hence,  region- 
specific  information  regarding  seismic  discriminants  is  vital  in  distinguishing  nuclear  explosions 
from  other  events.  Unfortunately,  relevant  ground-truth  data,  particularly  for  small  underground 
nuclear  explosions,  do  not  exist  for  most  regions.  In  addition,  it  has  yet  to  be  shown  that  a 
discrimination  rule,  established  in  a  region  for  which  data  exist,  can  be  transported  effectively  to  a 
new  region.  Thus,  in  most  cases,  seismic  event  identification,  within  the  context  of  CTBT  or  NPT 
(Non-Proliferation  Treaty)  monitoring,  is  a  problem  of  identifying  unusual  events,  i.e.,  outliers. 

Here  we  describe  a  technical  approach  to  categorize  seismic  events  with  these  and  other 
considerations  in  mind.  Our  objective  is  to  provide  an  automated  procedure  to  categorize  events, 
based  on  multivariate  analysis  of  features  derived  from  seismic  waveforms,  and  to  make  this 
information  available  in  a  database  at  the  Center  for  Monitoring  Research  (CMR).  Subscribers 
will  be  able  to  obtain  this  information,  to  aid  in  focussing  upon  events  of  particular  interest. 

Since  there  are  only  a  small  number  of  mining  blasts  that  occur  annually  above  magnitude  3,  our 
primary  concern  for  monitoring  above  this  threshold  is  to  be  able  to  distinguish  nuclear 
explosions  from  earthquakes.  Thus,  we  focus  on  this  case  here.  Monitoring  below  this  threshold 
poses  greater  difficulty  and  will  be  addressed  in  a  subsequent  report. 


In  Section  1.2,  we  describe  our  approach  which  first  determines  those  events  that  are  of  natural 
seismic  origin,  based  on  their  depth,  location  and  Ms-mb.  This  analysis  applies  to  only  those 
events  for  which  these  parameters  can  be  estimated,  currently  those  observed  at  teleseismic 
distances  by  many  stations.  Further  analysis  is  then  performed,  using  a  multivariate  statistical 
method  for  outlier  detection,  on  those  events  which  cannot  be  determined  to  be  of  natural  seismic 
origin  based  on  depth,  location  and  Ms-mb.  The  outlier  method  has  sufficient  generality  to  utilize 
any  combination  of  teleseismic  and  regional  discriminants  seen  at  single  or  multiple  stations.  The 
procedure  may  be  fully  automated  to  categorize  events  and  to  test  all  appropriate  assumptions  to 
ensure  validity  of  the  results.  Fisk  et  al.  (1993,  1994)  describe  the  methodology  in  detail  and 
provide  numerous  results  of  applications  to  seismic  data.  Gray  et  al.  (1994)  have  extended  the 
methodology  to  treat  data  values  which  are  commonly  missing  due  to  a  variety  of  physical  causes. 

The  method  can  be  used  to  perform  a  hypothesis  test,  with  a  controlled  false  alarm  rate,  as  to 
whether  an  event  is  an  outlier  or  not.  Fisk  et  al.  (1994)  applied  this  approach  to  seismic  events  in 
diverse  geological  regions,  recorded  by  the  ARCESS  and  GERESS  arrays  in  Norway  and 
Germany,  CDSN  (Chinese  Digital  Seismic  Network)  station  WMQ  in  China,  and  LNN 
(Livermore  NTS  Network)  stations  KNB  and  MNV  in  the  western  U.S.  Results  show  that  useful 
monitoring  can  be  performed  with  the  outlier-detection  approach,  currently  down  to  magnitude  3, 
for  regions  that  are  well-covered  by  at  least  one  seismic  station  or  array.  Overall,  264  of  290 
(91%)  explosions  were  identified  as  outliers  and  there  were  3  false  alarms  out  of  158  earthquakes 
(1.9%),  slightly  higher  than  the  target  rate  of  1%.  In  addition  to  the  diversity  of  the  regions,  these 
results  were  obtained  for  events  with  a  wide  range  of  epicentral  distances  and  magnitudes. 

Although  the  method  can  be  used  to  perform  a  hypothesis  test,  our  focus  here  is  to  use  the  outlier 
method  to  categorize  events  rather  than  providing  a  “yes/no”  decision.  In  Section  1.3,  we  define 
event  categories  based  on  the  discriminant  analysis  described  in  Section  1.2.  Categories  are 
intentionally  defined  in  a  non-judgemental  manner,  but  with  a  statistically  rigorous  interpretation. 
In  Section  1.4,  we  describe  the  data  sets  used  and  present  examples  of  the  categorization 
procedure  to  regional  data  sets  of  earthquakes  and  underground  nuclear  explosions  recorded  by 
the  ARCESS  array,  CDSN  station  WMQ,  and  LNN  stations  KNB  and  MNV. 
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1.2.  Overview  of  Technical  Approach 

Here  we  describe  our  approach  which  is  divided  into  two  distinct  steps.  The  objective  of  the  first 
step  is  to  screen  large  events  of  natural  seismicity,  based  on  the  context  in  which  the  event 
occurred,  namely,  event  depth  and  location.  We  also  use  Ms-mb  in  this  step  to  screen  earthquakes 
for  which  long-period  (20  second)  surface  waves  and  teleseismic  P  waves  are  measured.  Events 
that  are  determined  to  be  of  natural  seismicity  at  a  particular  confidence  level  are  not  processed 
further.  The  objective  of  the  second  step  is  to  categorize  the  remaining  events  using  a  multivariate 
statistical  outlier  method  (or  population  analysis)  which  measures  how  similar  or  different  the 
event  being  tested  is  to  other  events  seen  by  the  same  set  of  stations  or  arrays.  Details  of  these 
steps  are  presented  in  the  remainder  of  this  section. 

1.2.1.  Event  Screening 

The  analysis  in  this  step  is  based  on  the  physical  argument  that  events  that  are  deep  (e.g.,  deeper 
than  10  km)  cannot  be  of  man-made  origin.  Ms-mb  provides  a  robust  indicator  of  earthquakes, 
provided  they  are  not  too  deep,  for  events  above  mb  4.5.  Thus,  this  analysis  identifies  natural 
events  that  are  either  deep  or  shallow  but  with  Ms-mb  indicating  an  earthquake.  Events  located 
offshore  are  also  screened  and  can  be  investigated  more  reliably  using  hydroacoustic  techniques. 
Since  there  are  uncertainties  in  these  measurements,  the  thresholds  in  this  screening  process  are 
set  conservatively  to  maintain  a  high  level  of  confidence,  so  as  not  to  discard  events  of  possible 
man-made  origin  from  further  processing. 

Figure  1  illustrates  this  screening  process.  Event  parameters  required  to  perform  the  analysis  steps 
will  be  retrieved  from  REB  (Reviewed  Event  Bulletin)  database  at  the  CMR.  These  parameters 
will  include  an  estimate  of  event  depth  and  parameters  to  compute  its  confidence  interval,  the 
number  of  depth  phases  (e.g.,  pP)  detected,  an  estimate  of  epicentral  location  and  parameters  to 
compute  its  error  ellipse,  and  measurements  of  Ms  and  mb.  (Once  available,  hydroacoustic 
parameters  will  also  be  retrieved  for  offshore  events.)  Parameters  such  as  signal-to-noise  (SNR), 
the  number  of  reporting  stations,  and  quadrant  coverage  will  also  be  retrieved  from  the  database 
to  assess  the  quality  of  information.  The  analysis  then  proceeds  in  the  following  stages  in  terms  of 
these  parameters,  provided  they  could  be  measured  and  are  available  in  the  database. 

A)  Depth  Analysis.  The  first  step  is  to  determine  whether  the  entire  95%  confidence  interval  for 
depth  is  deeper  than  10  km.  If  it  is  and  at  least  one  depth  phase  (e.g.,  pP)  was  detected,  the  event 
is  categorized  as  a  deep  natural  event.  If  these  criteria  are  not  met,  the  event  is  processed  further. 
(Differences  in  P  and  S  travel  times,  once  computed  by  the  CMR,  may  also  be  used  to  better 
constrain  focal  depth.) 
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B)  Location  Analysis.  The  second  step  is  to  determine  whether  the  95%  confidence  error  ellipse 
of  epicentral  location  is  entirely  onshore,  entirely  offshore  or  overlapping.  If  the  error  ellipse  is 
entirely  offshore  and  no  hydroacoustic  bubble  pulse  is  detected,  the  event  is  categorized  as  a 
natural  offshore  event.  If  the  error  ellipse  is  either  entirely  or  partially  offshore  and  a  bubble  pulse 
is  detected,  the  event  is  categorized  as  a  offshore  explosion  of  unknown  (i.e.,  chemical  or  nuclear) 
source.  Remaining  events  are  processed  further. 

B)  Ms-mb  Analysis.  The  third  step  is  to  assess  whether  the  99%  Ms-mb  confidence  interval  is 
entirely  outside  the  nuclear  explosion  population,  based  on  previous  teleseismic  recordings  of 
known  nuclear  explosions.  Events  must  also  satisfy  the  conditions  that  SNR  of  the  long  period 
surface  waves  and  the  teleseismic  P  waves  are  both  greater  than  4  and  that  the  event  was  recorded 
by  stations  in  all  four  quadrants.  Events  which  satisfy  these  criteria  are  categorized  as 
earthquakes.  The  remainder  are  processed  further. 


Figure  1.  Schematic  of  first-cut  event  screening. 
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Note  that  the  criteria  stated  above  in  terms  of  confidence  levels,  SNR  thresholds  and  quadrant 
coverage  of  reporting  stations  are  currently  set  with  very  conservative  values  so  as  to  categorize 
natural  events  with  very  high  confidence,  while  performing  further  analysis  on  remaining  events. 
Criteria  in  terms  of  these  parameters  can  be  modified  as  necessary  or  desired. 

1.2.2.  Population  Analysis 

Due  to  current  limitations  in  estimating  depth,  location  and  Ms-mb  for  events  below  mb  4.5,  the 
remaining  events  will  be  predominantly  small.  (There  will  likely  also  be  some  large  events  which 
are  not  necessarily  deep,  not  necessarily  offshore,  and  lacking  high  confidence  evidence  regarding 
Ms-mb.)  Analysis  of  the  remaining  events  will  thus  be  based,  at  least  in  part,  on  high-frequency 
regional  discriminants  which  include  Pn/Lg  and  Pn/Sn.  Due  to  regional  variations  of  these 
discriminants  and  the  lack  of  calibration  data  for  nuclear  explosions  in  most  regions,  we  have 
adopted  an  outlier  procedure  to  characterize  events  observed  in  various  regions. 

In  the  population  (i.e.,  outlier)  analysis,  an  event  being  tested  is  compared  to  other  events  seen  at 
the  same  set  of  stations  or  arrays.  To  make  a  meaningful  comparison,  the  new  event  is  compared 
to  events  seen  in  the  same  region  (defined  by  distance  and  azimuth  relative  to  the  station)  when 
high-frequency  regional  discriminants  are  used.  In  most  practical  situations,  we  will  not  know  the 
identifications  of  these  events,  a  priori.  Thus,  an  assumption  is  made  that  the  number  of  new 
nuclear  tests  in  a  region  will  be  relatively  small  compared  to  the  number  of  earthquakes,  mining 
blasts  or  mine  tremors.  If  this  assumption  fails,  i.e.,  if  a  region  is  aseismic  and  with  no  mining 
activity,  then  any  new  event  would  be  suspicious,  warranting  further  investigation.  Since  there  are 
only  a  small  number  of  mining  blasts  that  occur  annually  above  mb  3,  our  primary  concern  for 
monitoring  above  this  threshold  is  to  distinguish  nuclear  explosions  from  earthquakes. 

A  standard  set  of  discriminants  is  used,  unless  we  have  information  about  particular  ones  that 
work  best  for  a  given  region.  For  the  examples  given  in  Section  1.4,  we  focus  on  Pn/Lg  and  Pn/Sn 
in  the  3-5,  4-6,  5-7  and  6-8  Hz  bands,  provided  measurements  are  available  for  the  events.  In 
general,  the  outlier  method  can  rigorously  include  any  discrete  or  continuous  discriminant. 
Regional  discriminants  are  corrected  for  attenuation  as  functions  of  type,  distance  and  frequency. 

The  likelihood  ratio  is  then  computed  for  each  event  and  categorized  in  terms  of  its  value  relative 
to  the  distribution  of  the  likelihood  ratio  for  the  remaining  events.  (Technical  details  of  the  outlier 
test  and  previous  applications  are  provided  by  Baek  et  al.,  1992;  Fisk  et  al.,  1993,  1994;  Fisk  and 
Gray,  1993;  Fisk,  1993.) 
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The  likelihood  ratio  is  given  by 

^  _  max  L(parameters  I  data;  under  null  hypothesis) 

max  L(parameters  I  data;  under  alternative  hypothesis)  ’ 

where  the  numerator  is  computed  under  the  null  hypothesis  being  tested,  given  the  data,  and  the 
denominator  is  computed  under  the  alternative  hypothesis.  For  the  outlier  test,  the  null  hypothesis 
is  that  the  event  being  tested  belongs  to  the  same  population  as  the  remainder  of  the  events.  The 
alternative  hypothesis  is  that  it  is  not  a  member.  Since  the  true  parameters  (e.g.,  the  mean  and 
covariance  matrix  of  the  discriminants)  of  the  likelihood  functions  are  unknown,  maximum 
likelihood  estimates  are  computed  from  the  data,  subject  to  the  particular  hypothesis,  and  inserted 
into  the  likelihood  functions.  This  yields  the  ratio  of  maximized  likelihoods.  The  resulting 
expression  for  X  combines  multivariate  discriminant  data  for  the  event  being  tested  and  the 
reference  events  into  a  univariate  expression.  This  provides  a  straightforward  metric  with  which 
to  perform  a  hypothesis  test,  to  rank  and/or  categorize  events. 

Small  values  of  X  indicate  that  the  null  hypothesis  should  be  rejected.  To  quantify  what  is  meant 
by  “small,”  the  distribution  of  X  is  needed.  Under  the  assumption  that  the  discriminants  are 
multivariate  normal,  the  distribution  of  X  may  be  obtained  in  closed  form  in  terms  of  Hotelling’s 
T^-distribution  or,  equivalently,  the  F-distribution.  In  general,  the  distribution  of  X  is  quite 
complicated,  often  with  no  closed  form  expression,  depending  on  the  multivariate  discriminant 
distribution.  To  estimate  it  empirically  under  more  general  assumptions,  we  use  the  bootstrap 
technique  (Efron,  1979)  to  generate  random  samples  from  the  distribution  of  discriminants  for  the 
reference  events,  and  insert  bootstrapped  data  in  the  likelihood  ratio  for  many  samples.  From  this 
distribution  of  X,  a  threshold,  X^ ,  may  be  set  so  that  the  test  has  a  specified  false  alarm  rate.  That 
is,  X^  is  set  such  that  =  a,  where  a  is  the  significance  level.  Events  whose 

likelihood  ratio  are  less  than  X^  are  considered  outliers  at  the  specified  significance  level. 

To  formulate  the  results  in  terms  of  event  categories  rather  than  providing  a  rigid  “yes/no” 
judgement,  we  define  the  categories  in  terms  of  thresholds  for  various  significance  levels.  That  is, 
we  define  the  categories  in  terms  of  a  range  of  probabilities,  based  on  the  distribution  of  X,  that  an 
earthquake  in  a  particular  region  would  fall  in  a  given  category.  Thus  an  event  is  placed  in  the  ith 
category  if  its  value  of  the  likelihood  ratio,  X^. ,  satisfies  ,  where  X„  is  set  such 

that  P|^X<X^  |HqJ  =  a  - .  Equivalently,  an  event  falls  in  the  /th  category  if  ^OL-, 

where  =  P[X<  Xq|Hq]  is  the  probability  that  an  earthquake  would  have  a  value  of  X  less 
than  the  value  for  the  event  being  tested.  If  an  event  falls  in  a  category  corresponding  to  a  small 
value  of  a^.,  the  probability  that  this  event  is  an  earthquake  is  likewise  small.  In  Section  1.3  we 
define  event  categories  in  terms  of  the  and  provide  further  interpretation. 
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Note  that  if  the  value  of  the  likelihood  ratio  for  an  event  being  tested  is  one,  discriminant  values 
for  this  event  are  identical  to  the  sample  discriminant  means  based  on  the  other  events.  Thus,  in 
this  case,  the  event  being  tested  is  entirely  consistent  with  the  remaining  population.  Any 
departure  of  the  discriminant  values  from  the  sample  means  leads  to  smaller  values  of  A,.  The 
likelihood  ratio  does  not  differentiate  between  events  that  have  abnormally  small  versus  large 
discriminant  values.  It  is  a  measure  of  the  absolute  distance  of  the  discriminant  values  for  the  new 
event  from  the  means,  based  on  the  other  events,  as  compared  to  the  covariance  matrix.  This 
allows  for  cases  in  which  an  event  could  rightfully  be  called  an  outlier,  but  with  discriminant 
values  that  are  even  more  inconsistent  with  explosions  than  with  earthquakes.  While  it  is  useful  to 
flag  such  peculiar  events  for  analyst  review,  this  could  lead  to  confusion  in  an  automated  setting. 
To  alleviate  potential  confusion,  we  define  in  the  following  section  a  category  which  corresponds 
to  events  for  which  all  discriminant  values  are  on  the  side  of  the  discriminant  means,  based  on  the 
remaining  events,  away  from  values  for  explosions. 

1.3.  Definition  of  Event  Categories 

Based  on  the  approach  just  described,  we  now  define  ten  event  categories.  Category  1 
corresponds  to  events  which  are  determined  to  be  of  natural  seismic  origin  based  on  depth  and 
detection  of  at  least  one  depth  phase,  location  and  lack  of  a  hydroacoustic  bubble  pulse  (for  events 
offshore),  and/or  Ms-mb.  Category  2  is  defined  by  events  whose  Pn/Lg  and  Pn/Sn  values  are  all 
less  than  the  means,  based  on  the  reference  events.  Categories  3-10  are  defined  in  terms  of 
probabilities,  a  - ,  which  set  the  criteria  A  <  A  <  A  .  The  probability  that  an  event  would  fall  in 
the  zth  category  if  it  was  an  earthquake  is  Prob[/]  =  <A<A^|HqJ  = 

(Note  that  for  a  fixed  set  of  a-,  the  A^^^  depend  on  the  multivariate  distribution  of  the 
discriminants  for  events  in  various  regions,  as  well  as  on  the  sample  size  of  reference  events.  The 
outlier  methodology  automatically  adapts  the  category  thresholds  to  the  quality  and  quantity  of 
data  in  a  given  region  to  maintain  a  rigorous  interpretation  in  terms  of  probabilities  that  an 
earthquake  fall  in  a  particular  category.)  Table  1  summarizes  definitions  of  the  event  categories. 

Figure  2  shows  how  Categories  3- 10  are  defined  in  terms  of  the  cumulative  distribution  function 
(cdf)  of  the  log  likelihood  ratio,  under  the  null  hypothesis  that  the  event  being  tested  belongs  to 
the  same  group  as  the  reference  events.  The  categories  are  defined  by  the  lines  which  intersect  the 
cdf  at  particular  values,  namely,  0.5,  0.1,  0.01, 0.001, 0.0001,  0.00001, 0.000001.  Category  3,  for 
example,  contains  50%  of  the  earthquakes  which  occur  in  a  given  region  and  are  also  the  most 
typical  of  the  events  recorded  there.  Events  that  fall  in  the  other  categories  are  progressively  less 
consistent  with  the  earthquake  population  in  the  region.  Events  that  fall  in  Category  10,  for 
example,  have  a  probability  less  than  or  equal  to  0.000001  of  being  an  earthquake  in  this  region. 
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Table  1.  Summary  of  Event  Category  Definitions. 


Category 

Description 

Category  1: 

Category  1(a) 

Category  1(b) 

Category  1(c) 

Very  high  confidence  natural  event  (based  on  event  context  and  Ms-mb) 

•  95%  depth  confidence  interval  >  5  km  and  depth  phase  pP  detected 

•  95%  location  error  ellipse  entirely  offshore  and  no  bubble  pulse  detected 

•  99%  Ms-mb  confidence  interval  is  entirely  outside  explosion  population 

Category  2 

Events  with  Pn/Lg  &  Pn/Sn  <  means,  based  on  the  remaining  events  in  the  region 

Category  3 

0.5  <Py  <1.0;  Prob  =  0.5  that  an  earthquake  will  fall  in  this  category 

Category  4 

0.1  <  P.  <  0.5  ;  Prob  =  0.4  that  an  earthquake  will  fall  in  this  category 

Category  5 

0.0\  <Py  <  0.1  ;  Prob  =  0.09  that  an  earthquake  will  fall  in  this  category 

Category  6 

0.001  <P^  ^  0.01  ;  Prob  =  0.009  that  an  earthquake  will  fall  in  this  category 

Category  7 

0.0001  <  P.  <  0.001  ;  Prob  =  0.0009  that  an  earthquake  will  fall  in  this  category 

Category  8 

0.00001  <  Pi  ^  0.0001;  Prob  =  0.00009  that  an  earthquake  will  fall  in  this  category 

Category  9 

0.000001  <P-i  <  0.00001  ;  Prob  =  0.000009  that  an  earthquake  will  fall  in  this  category 

Category  10 

P.  <  0.000001  ;  Prob  =  0.000001  that  an  earthquake  will  fall  in  this  category 

1.4.  Examples  of  Event  Categorization 

To  illustrate  the  categorization  procedure,  we  applied  it  to  earthquakes  and  nuclear  explosions 
recorded  by  the  ARCESS  array,  CDSN  station  WMQ,  and  LNN  stations  KNB  and  MNV. 
ARCESS  events  include  24  earthquakes  near  Steigen,  Norway,  5  earthquakes  near  Spitsbergen, 
and  3  underground  nuclear  explosions  at  the  Novaya  Zemlya  test  site.  WMQ  events  include  23 
earthquakes  in  China  and  nearby  countries,  16  nuclear  explosions  in  Kazakhstan,  and  1  nuclear 
explosion  at  the  Lop  Nor  test  site  in  China.  Data  for  these  events  were  provided  by  Baumgardt 
(1993).  Patton  and  Walter  (1994)  also  provided  us  with  seismic  data  from  stations  KNB  and  MNV 
for  a  total  of  76  earthquakes,  141  nuclear  explosions,  and  1  contained  1  kt  chemical  explosion  at 
NTS.  The  chemical  explosion  was  the  September  1993  Non-Proliferation  Experiment  (NPE).  Due 
to  SNR  limitations  for  some  events,  Pn/Lg  measurements  were  computed  for  59  earthquakes  and 
89  explosions  of  those  detected  by  KNB.  Similar  measurements  were  computed  for  37 
earthquakes  and  78  explosions  of  those  detected  by  MNV. 
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Figure  2,  Relationship  of  Categories  3-10  to  the  cumulative  probability  of  the  log  likelihood  ratio  when  the 
null  hypothesis  is  true. 


Table  2  summarizes  the  events,  their  epicentral  distances  and  magnitudes.  Figure  3  depicts 
locations  of  the  seismic  stations,  arrays,  and  events  considered  in  this  study  (as  well  as  others  that 
were  not  considered  here).  The  combined  data  set  represents  events  in  diverse  geological  regions, 
seen  at  a  wide  range  of  regional  distances  (100  to  1300  km),  and  with  a  wide  range  of  magnitudes 
(1.0  to  6.1).  Fisk  et  al.  (1994)  provide  a  more  detailed  discussion  of  these  data  sets. 

Discriminants  used  in  this  study  include  Pn/Lg  and  Pn/Sn  in  3-5, 4-6,  5-7,  6-8  Hz  bands.  Of  these, 
only  Pn/Lg  measurements  in  the  6-8  Hz  band  were  provided  for  events  recorded  by  KNB  and 
MNV.  Also,  Sn  was  measured  for  only  2  of  17  explosions  recorded  by  WMQ.  Thus,  only  Pn/Lg  in 
the  listed  frequency  bands  was  used  for  this  station.  Baumgardt  et  al.  (1992)  describe  how  these 
discriminants  are  computed  from  seismic  waveforms  for  the  ARCESS  and  WMQ  data  sets,  while 
Walter  et  al.  (1994)  provide  similar  descriptions  for  the  KNB  and  MNV  data  sets.  We  did  not 
receive  depth  or  Ms-mb  measurements  for  these  events  and  none  had  location  error  ellipses 
entirely  offshore.  Thus,  none  of  these  events  could  be  placed  in  Category  1. 
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Table  2.  Summary  of  seismic  data  sets  used  in  this  study. 


Array/Station 

Events 

Distance  (km) 

Magnitude 

ARCESS 

24  Steigen  EQs 

385-480 

1. 0-3.2 

5  Spitsbergen  EQs 

795-1320 

1. 5-2.9 

3  NZ  EXs 

1100 

>3.9 

WMQ 

23  EQs 

100-1100 

4.2-5.9 

16  Kazakh  EXs 

950 

4.8-6. 1 

1  Lop  Nor  EX 

240 

4.7 

KNB 

59  NTS  EQs 

295-310 

2. 1-5.9 

89  NTS  EXs 

280-315 

2.4-5.5 

MNV 

37  NTS  EQs 

250-260 

2.2-5.9 

Figure  3.  Locations  of  ARCESS,  CDSN  station  WMQ,  LNN  stations  KNB  and  MNV,  and  the  seismic  events 

used  in  this  study. 


1.4.1.  Categorization  Results  for  WMQ  Events 


Figure  4  shows  locations  of  CDSN  station  WMQ,  16  nuclear  explosions  at  the  Balapan  test  site  in 
Kazakhstan,  the  29  September  1988  (880929)  nuclear  explosion  at  the  Lop  Nor  test  site  in  China, 
and  23  earthquakes  scattered  mostly  around  the  northwest  portion  of  China  and  western 
Mongolia.  These  events  ranged  in  distance  from  100  to  1100  km  and  from  magnitude  4.2  to  6.1. 
Figure  5  shows  distance-corrected  Pn/Lg  values  in  nine  frequency  bands  from  0.5  to  16  Hz.  Of 
these,  the  3-5,  4-6,  5-7,  and  6-8  Hz  bands  were  used  in  the  outlier  analysis. 


Figure  4.  Locations  of  CDSN  station  WMQ,  16  nuclear  explosions  in  Kazakhstan,  the  880929  nuclear 
explosion  at  Lop  Nor  and  23  earthquakes  in  China  and  nearby  countries. 


Figure  6  shows  categorization  results  for  the  23  earthquakes  and  17  nuclear  explosions  recorded 
by  station  WMQ.  The  categorized  events  are  labelled  by  their  origin  identification  (GRID) 
numbers.  ORIDs  shown  in  white  correspond  to  actual  earthquakes,  while  those  shown  in  black 
correspond  to  underground  nuclear  detonations.  For  this  case  all  17  nuclear  explosions  fall  in 
Category  10.  There  are  7  earthquakes  in  Category  2  with  Pn/Lg  values  in  all  four  bands  less  than 
the  corresponding  means  computed  from  the  remaining  earthquakes.  Seven  other  earthquakes  fall 
in  Category  3,  5  in  Category  4,  3  in  Category  5  and  1  in  Category  6. 
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Figure  5.  Distance-Corrected  Pn/Lg  in  nine  frequency  bands  for  23  earthquakes  and  17  nuclear  explosions 
recorded  by  WMQ. 


For  comparison,  Figure  7  shows  results  of  the  outlier  hypothesis  test  for  the  same  events.  The 
curve  shown  is  of  the  probability  density  function  (pdf)  of  the  log  likelihood  ratio  when  the  event 
being  tested  is  from  the  same  group  as  the  earthquake  training  set.  The  vertical  line  represents  the 
threshold  of  the  test  for  the  significance  level  listed  in  the  lowest  legend.  Events  whose  likelihood 
ratio  are  less  than  the  threshold  are  identified  as  outliers  at  the  corresponding  significance  level. 
The  triangles  and  circles  depict  values  of  the  likelihood  ratio  for  the  explosions  and  earthquakes 
being  individually  tested,  respectively.  Figure  7  shows  that  all  17  nuclear  explosions  are  flagged 
as  outliers  of  the  earthquake  group  at  0.01  significance  level.  There  was  one  false  alarm  out  of  the 
23  earthquakes.  This  earthquake  corresponds  to  ORID=393656  shown  in  Category  6  of  Figure  6. 


Density  Events  (ORIDs) 


1.4.2.  Categorization  Results  for  ARCESS  Events 

Events  considered  for  ARCESS  include  3  nuclear  explosions  at  the  Novaya  Zemlya  test  site,  24 
earthquakes  near  Steigen,  Norway,  and  5  earthquakes  near  Spitsbergen.  Figure  8  shows  the 
locations  of  these  events  (as  well  as  quarry  blasts  on  the  Kola  Peninsula,  not  considered  here)  and 
the  ARCESS  array.  Table  2  summarizes  magnitude  and  distance  information  for  these  events. 
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Figure  8.  Locations  of  the  ARCESS  array,  3  nuclear  explosions  on  Novaya  Zemlya,  5  Spitsbergen  and  24 
Steigen  earthquakes. 

Figure  9  shows  scatter  plots  of  Pn/Sn  measurements  in  six  frequency  bands  for  these  and  other 
events  after  applying  distance  corrections.  The  upper  legend  associates  the  marker  type  with  the 
event  type.  This  plot  shows  that  there  is  good  separation  between  the  earthquake  and  nuclear 
explosion  groups  in  the  bands  above  4  Hz.  Pn/Lg  measurements  were  not  available  for  the 
Novaya  Zemlya  events,  nor  the  5  earthquakes  near  Spitsbergen,  since  Eg  does  not  propagate 
efficiently  beneath  the  Barents  Sea. 

Figure  10  shows  categorization  results  for  the  29  earthquakes  and  3  Novaya  Zemlya  nuclear 
explosions,  based  on  Pn/Sn  recordings  at  the  ARCESS  array.  Figure  10  shows  that  all  3  nuclear 
explosions  fall  in  Category  10.  Applying  the  outlier  hypothesis  test,  Fisk  et  al.  (1994)  found  that, 
at  0.01  significance  level,  the  3  Novaya  Zemlya  explosions  were  identified  as  outliers  of  the 
Steigen/Spitsbergen  earthquake  group  and  there  were  no  false  alarms. 
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Distance-corrected  Pn/Sn  measurements  in  six  frequency  bands  for  events  recorded 


1  ^  i  i  i  ^  r 

Discriminant(s): 
Pn/Sn(max;3“5 
Pn/5n(max;4-6 
Pn/5n(max;5-7 
Pn/Sn(max;6-8 


Training  Set;  Steigen/Spitsbergen_ARAO_EQ 

Test  Set  1:  Steigen/Spitsbergen.ARAO.EQ 

Test  Set  2:  NovayaZemlya_ARAO_EX 


1.4.3.  Categorization  Results  for  KNB  and  MNV  Events 

To  illustrate  how  the  categorization  procedure  applies  to  events  detected  by  more  than  one  station, 
we  use  data  from  both  KNB  and  MNV  to  categorize  the  earthquakes  and  explosions  which 
occurred  at  the  Nevada  Test  Site  (NTS).  Figure  11  shows  a  map  of  the  southwestern  U.S. 
depicting  locations  of  LNN  stations  KNB  and  MNV  relative  to  NTS.  On  the  left  is  an  enlarged 
map  of  NTS  showing  epicenters  of  the  nuclear  explosions  and  earthquakes  used  in  this  study.  The 
earthquakes,  ranging  from  magnitude  (ML)  2.1  to  5.9,  were  clustered  in  three  main  locations; 
Little  Skull  Mountain,  Rock  Valley,  and  one  at  Massachusetts  Mountain.  The  explosions  ranged 
in  magnitude  from  ML  2.4  to  5.5  and  in  depth  from  200  to  nearly  700  meters,  in  media  with  a 
relatively  wide  range  of  properties.  To  conform  more  closely  to  the  event  definition  currently 
adopted  by  the  CMR  for  GSETT-3,  we  consider  only  those  events  for  which  a  P  phase  was 
detected  at  both  stations.  This  definition  is  satisfied  by  35  earthquakes  and  69  nuclear  explosions. 


Figure  11.  Locations  of  LNN  stations  KNB  and  MNV  (among  others)  relative  to  NTS.  On  the  left  is  an 
enlarged  map  of  NTS  showing  epicenters  of  the  nuclear  explosions  and  earthquakes. 

Figures  12  and  13  show  Pn/Lg  measurements  in  the  6-8  Hz  band  versus  ML(coda)  for  the  events 
recorded  by  KNB  and  MNV,  respectively.  (Note  that  Sn  measurements  were  not  available  since 
Sn  does  not  propagate  efficiently  in  the  western  U.S.)  The  explosions  are  separated  by  those 
which  were  detonated  in  media  with  high  versus  low  gas  porosity  (GP)  and  the  earthquakes  are 
separated  into  subregions  in  which  they  occurred.  The  legend  associates  the  marker  types  with  the 


16 


various  events.  Pn/Lg  at  MNV  provides  the  best  discrimination  of  earthquakes  and  explosions  in 
this  region,  while  there  is  more  overlap  at  KNB,  particularly  for  the  shallow  (1-3  km  deep)  Rock 
Valley  earthquakes.  Walter  et  al.  (1994)  noted  that  Pn/Lg  exhibits  differing  values  for  shallow 
than  for  deeper  earthquakes  at  KNB,  possibly  due  to  radiation  pattern  or  path  effects.  There  is  no 
significant  difference  between  Pn/Lg  values  for  explosions  in  media  of  high  versus  low  gas 
porosity.  Figures  12  and  13  show  that  Pn/Lg  in  the  6-8  Hz  band  exhibits  some  dependence  on 
magnitude,  ML(coda),  at  both  stations,  although  this  is  actually  a  signal-to-noise  effect. 


lO' 


00 

1 

CD 


CL 


10’ 


Pn/Lg(6-8  Hz)  at  KNB 


Explosions  High  GP 
X  Explosions  Low  GP 

K  N'PE  Chemical  Explosion 

O  Skull  Earthquakes 

D  Rock  Earthquakes 

^  Mass  MLn  Earthauake 


X 


X  X 


-k- 

c 


-X- 

X 


-X 


© 


*  X 
X  % 


o  o 


4 

ML 


Figure  12.  Pn/Lg  measurements  in  the  6-8  Hz  band  versus  ML(coda)  for  events  recorded  by  KNB. 

Figure  14  shows  categorization  results  for  these  events  using  Pn/Lg(6-8  Hz)  measurements  at 
both  stations.  Performance  at  these  stations  is  significantly  worse  than  at  either  ARCESS  or 
WMQ,  where  more  discriminants  are  available.  Note  that  for  the  outlier  hypothesis  test  at  0.01 
significance  level,  59  of  69  (86%)  explosions  are  identified  as  outliers  and  there  are  no  false 
alarms.  Events  in  Categories  6-10  are  those  identified  as  outliers  by  the  hypothesis  test  at  0.01 
significance  level.  Using  Pn/Lg(6-8  Hz)  for  the  same  set  of  events  at  the  individual  stations,  63  of 
69  (91%)  explosions  are  identified  as  outliers  at  MNV  and  20  of  69  (29%)  explosions  are 
identified  as  outliers  at  KNB  and  there  are  no  false  alarms  at  either  station.  Thus,  combining 
evidence  from  the  two  stations  is  almost  as  effective  as  only  using  the  best  station  (MNV)  and  is 
significantly  better  than  only  using  the  worst  station  (KNB).  Note  that  in  most  practical 
monitoring  situations  we  will  not  necessarily  know  which  station  provides  better  discrimination. 
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These  results  also  show  that  additional  discriminants  are  needed,  for  this  region  in  particular.  In  a 
previous  study,  Fisk  et  al.  (1994)  used  a  spectral  ratio  of  Lg  coda,  in  addition  to  Pn/Lg(6-8  Hz),  at 
the  individual  stations.  They  found  that  95%  and  80%  of  the  explosions  were  identified  as  outliers 
at  MNV  and  KNB,  respectively,  as  compared  to  91%  and  29%  using  only  Pn/Lg.  Although  Lg 
spectral  ratios  discriminate  in  some  regions  (e.g.,  the  western  U.S.  and  Germany),  they  do  not  in 
others  (e.g.,  Scandinavia  and  Eurasia)  (Fisk  et  al.,  1994).  Thus,  we  are  hesitant  to  use  them  until 
further  physical  understanding  of  where  and  why  they  discriminate  is  obtained.  Also,  Fisk  et  al. 
(1995)  included  Pg/Lg(6-8  Hz)  in  the  outlier  analysis  and  found  further  improvement. 

1.5.  Summary 

We  have  provided  an  event  categorization  procedure  that  focussed  here  on  categorizing  events 
which  satisfy  the  event  definition  currently  used  at  the  CMR  in  terms  of  a  minimum  number  of 
phase  detections  at  the  Alpha  stations.  This  definition  effectively  restricts  analysis  to  events  above 
mb  3,  possibly  above  mb  3.5  (Carter,  1995),  for  which  the  primary  concern  is  distinguishing 
underground  nuclear  explosions  from  earthquakes. 

Confidence  intervals  for  universal  discriminants  (such  as  depth,  location  and  Ms-mb)  are  used  to 
categorize  events  when  such  measurements  can  be  obtained.  These  rules  define  Category  1  which 
consist  of  high  confidence  natural  events,  typically  those  above  mb  4.5.  Events,  for  which 
regionally-varying  discriminants  such  as  Pn/Lg  and  Pn/Sn  are  used,  are  categorized  using  a 
multivariate  population  (i.e.,  outlier)  analysis  which  compares  new  events  to  others  seen  in  the 
same  region  by  the  same  set  of  stations  or  arrays.  It  is  assumed  that  the  number  of  new  nuclear 
tests  in  a  region  will  be  small  compared  to  the  number  of  earthquakes  in  that  region.  Using  the 
distribution  of  the  likelihood  ratio.  Categories  3-10  are  defined  in  terms  of  the  cumulative 
probability  that  an  earthquake  in  a  given  region  would  fall  in  a  particular  category.  Since  the 
likelihood  ratio  does  not  differentiate  between  events  with  discriminant  values  that  are  higher  or 
lower  than  the  discriminant  means,  Category  2  is  reserved  for  those  events  with  all  of  their 
discriminant  values  on  the  side  of  the  means  away  from  values  for  explosions. 

The  examples  in  Section  1 .4  show  that  all  nuclear  explosions  recorded  by  either  station  WMQ  or 
the  ARCESS  array  fall  in  Category  10.  Using  a  single  discriminant,  Pn/Lg(6-8  Hz),  measured  at 
both  KNB  and  MNV,  there  were  no  Category  10  events,  although  59  of  69  (89%)  explosions  were 
in  Category  6  or  higher.  This  case  illustrates  how  evidence  from  multiple  stations  can  be 
combined,  as  well  as  the  need  for  additional  discriminants,  particularly  in  this  region.  To  extend 
this  categorization  procedure  to  even  smaller  events  (mb  <  3),  future  effort  will  focus  on 
investigating  additional  discriminants,  techniques  to  discriminate  single  explosions  from  ripple- 
fired  mining  blasts,  and  on  defining  additional  categories  for  mining  blasts  and  mine  tremors. 
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2.  Preliminary  Identification  Analysis  of  the  950105  Urals  Event 
2.1.  Introduction 

Based  on  origin  analysis  at  the  CMR,  a  seismic  event  of  mb  4.35  occurred  at  origin  time  01/05/95 
12:46:01  (GMT),  latitude  59.52  and  longitude  56.31  in  the  Southern  Ural  Mountains  region.  (We 
will  refer  to  this  as  the  950105  event.)  It  was  a  shallow  event  which  occurred  in  an  area  of  low 
seismicity  and  near  where  several  PNEs  had  been  conducted.  Of  further  interest  was  the  fact  that 
it  had  a  relatively  small  Ms:mb  value,  inconsistent  with  typical  values  for  shallow  earthquakes. 

A  press  release  by  the  Moscow  ITAR-TASS  World  Service,  6  January  1995,  reported  a  statement 
by  the  Russian  Federation’s  State  Technical  Inspectorate  that  this  event  was  a  tectonic  shock  at  Pit 
Number  2  of  the  Silvinit  Joint-Stock  Company,  caused  by  an  earthquake  with  its  epicenter  near 
the  settlement  of  Chashkino  in  Perm  Oblast.  The  announcement  also  indicates  that  the  tectonic 
shock  led  to  the  destruction  of  ventilation  systems  at  the  mine  and  to  considerable  emissions  of 
explosive  gases  -  methane  and  hydrogen  -  which  subsequently  ignited  and  exploded. 

The  objective  of  our  analysis  is  to  independently  assess  the  identification  of  this  event  by 
comparing  its  seismic  data,  recorded  by  the  ARCESS  and  KVAR  arrays  and  stations  ARU  and 
OBN,  to  that  of  previous  earthquakes  and  nuclear  explosions.  Since  the  950105  event  had  a 
magnitude  of  mb  4.35,  considerably  larger  than  would  be  produced  by  a  large  mining  blast,  we 
focus  here  on  identifying  this  event  as  either  an  earthquake  or  a  nuclear  explosion.  (Walter,  1995, 
and  others  have  considered  other  alternatives  such  as  a  mine  collapse.) 

In  Section  2.2  we  describe  seismic  data  sets  which  are  currently  available  to  us,  consisting  of 
earthquakes  and  nuclear  explosions  in  several  regions,  recorded  by  the  ARCESS  array  in  Norway 
and  station  WMQ  in  China.  We  compare  high-frequency  Pn/Lg  and  Pn/Sn  measurements  for  the 
950105  event  to  equivalent  measurements  for  these  reference  data  sets.  Measurements  are 
corrected  for  instrument  response  and  distance  effects.  Since  measurements  for  the  950105  event 
are  not  available  for  WMQ,  comparisons  to  events  recorded  by  WMQ  are  noted  with  the  caveat 
that  there  may  be  significant  path  and  station  differences.  Also,  although  the  950105  event  was 
recorded  at  the  ARCESS  array,  its  propagation  distance  and  azimuth  are  considerably  different 
than  for  the  ARCESS  reference  events.  We  provide  conclusions  regarding  the  identification  of  the 
950105  event,  based  on  these  reference  data  sets,  with  these  caveats  in  mind. 

In  Section  2.3  we  categorize  the  950105  event  using  our  multivariate  outlier  analysis  in  which  we 
compare  high-frequency  Pn/Sn  measurements  at  ARCESS  to  those  for  earthquakes  and  nuclear 
explosions  recorded  by  the  same  array.  In  Section  2.4  we  provide  preliminary  conclusions 
regarding  the  identification  of  this  event. 
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2.2.  Data 


Figure  15  shows  the  location  of  the  950105  event  (labeled  by  its  origin  identification  number, 
orid=273228),  as  well  as  locations  of  the  ARCESS  and  KVAR  arrays,  stations  ARU  and  OBN, 
and  the  reference  events  used  in  this  study.  Epicentral  distances  of  the  950105  event  to  ARCESS, 
KVAR,  ARU  and  OBN  are  1812,  1960,  369,  and  1275  km,  respectively.  Reference  events  for 
ARCESS  include  3  Novaya  Zemlya  nuclear  explosions  (triangles),  24  earthquakes  in  the  Steigen 
region  of  Norway  (open  circles)  and  5  earthquakes  near  Spitsbergen  (solid  circles).  Reference 
events  for  WMQ  include  16  nuclear  explosions  in  Kazakhstan  (triangles),  1  nuclear  explosion  at 
the  Lop  Nor  test  site  in  China  (triangle),  and  23  earthquakes  in  China  and  nearby  countries  (solid 
circles).  In  addition,  we  provide  comparisons  to  10  Vogtland  earthquakes  (open  circles)  recorded 
by  the  GERESS  array  and  to  16  PNEs  (asterisks)  recorded  by  the  NORSAR  array.  Table  3 
provides  a  summary  of  these  events,  their  epicentral  distances  and  magnitudes. 


Figure  15.  Locations  of  seismic  stations,  arrays  and  events  used  in  our  analysis  of  the  950105  Southern  Urals 
event  (ORID=273228). 
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Table  3.  Summary  of  seismic  data  sets  used  in  the  analysis  of  the  950105  Urals  Event 


Array/Station 

Events 

Distance  (km) 

Magnitude 

ARCESS 

24  Steigen  EQs 

385-480 

1. 0-3.2 

5  Spitsbergen  EQs 

795-1320 

1 .5-2.9 

3  NZ  EXs 

1 100 

>3.9 

GERESS 

10  Vogt) and  EQs 

140-260 

1. 4-3.2 

WMQ 

23  EQs 

100-1100 

4.2-5.9 

16  Kazakh  EXs 

950 

4.8-6. 1 

1  Lop  Nor  EX 

240 

4.7 

NORSAR 

16PNES 

1320-4210 

4.6-6. 1 

2.2.1.  Amplitude  Ratios  Prior  to  Applying  Distance  Corrections 

Figure  16  shows  Pn/Lg  maximum  amplitude  measurement  in  the  6-8  Hz  band  versus  distance  for 
the  950105  and  reference  events.  These  values  are  uncorrected  for  distance  effects.  Pn/Lg 
measurements  for  the  950105  event  are  provided  for  the  ARU  and  OBN  recordings;  no  Pn/Lg 
measurements  were  obtained  for  ARCESS  and  KVAR.  The  legend  associates  the  marker  types  to 
the  various  events.  This  figure  shows  that  the  Pn/Lg(6-8  Hz)  values  for  the  950105  event  are 
considerably  different  than  those  for  nuclear  explosions  and  consistent  with  those  for  earthquakes. 

Figure  17  shows  similar  results  for  Pn/Sn(6-8  Hz)  measurements,  prior  to  applying  distance 
corrections.  In  this  case,  Pn/Sn  measurements  for  the  950105  event  are  available  at  ARCESS, 
KVAR,  ARU  and  OBN.  Note  that  Pn/Sn  measurements  were  obtained  for  only  1  of  the  nuclear 
explosions  recorded  by  WMQ.  As  for  Pn/Lg(6-8  Hz),  Pn/Sn(6-8  Hz)  values  at  all  four  stations  or 
arrays  for  the  950105  event  are  more  consistent  with  values  for  earthquakes. 

Eigure  18  shows  ARCESS  Pn/Sn  measurements  in  nine  frequency  bands  on  the  sz  channel,  before 
applying  distance  corrections,  for  3  Novaya  Zemlya  nuclear  explosions,  24  Steigen  earthquakes 
and  the  950105  Urals  event.  The  950105  event  compares  differently  to  the  earthquakes  and 
explosions  in  different  bands.  Note  that  the  950105  event  was  over  1800  km  from  ARCESS  and 
exhibits  considerable  variation  with  frequency,  likely  due  to  attenuation  effects.  In  Section  2.2.2 
we  apply  distance  corrections  to  make  a  more  meaningful  comparison. 
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Distance  (km) 


Figure  16.  Uncorrected  Pn/Lg  in  the  6-8  Hz  band  versus  distance  for  earthquakes,  nuclear  explosions,  and 
the  950105  Urals  event. 


Distance  (km) 


Figure  17.  Uncorrected  Pn/Sn  in  the  6-8  Hz  band  versus  distance  for  earthquakes,  nuclear  explosions,  and 
the  950105  Urals  event. 


cNj^'^LricjDf^ccoto 

I  I  i  !  I  I  I  I  I 

iDOlOOOOOOO 
O  trv  r\'  lH  CO  cd  o6 

Figure  18.  Uncorrected  Pn/Sn  measurements  in  nine  frequency  bands  for  3  Novaya  Zemlya  nuclear 
explosions,  24  Steigen  earthquakes  and  the  950105  event  recorded  by  ARCESS. 

Figure  19  shows  Pn/Lg  measurements  in  eight  frequency  bands  on  bz  channels  at  stations  ARU 
and  OBN  for  the  950105  event  before  applying  distance  corrections.  Also  shown  are  Pn/Lg 
values  for  23  earthquakes  and  17  nuclear  explosions  recorded  by  station  WMQ.  In  all  bands 
above  3  Hz,  the  Pn/Lg  values  for  the  950105  event  are  consistent  with  those  for  the  earthquakes 
and  considerably  different  than  those  for  the  nuclear  explosions.  Figure  20  shows  a  similar  plot 
for  Pn/Sn  measurements.  There  were  no  Pn/Sn  measurements  for  the  nuclear  explosion  recorded 
by  WMQ  on  the  bz  channel.  Figure  20  shows  that  the  Pn/Sn  values  for  the  950105  event  are, 
however,  consistent  with  those  for  the  earthquakes  in  all  frequency  bands. 

2.2.2.  Amplitude  Ratios  After  Applying  Distance  Corrections 

Figure  21  shows  distance-corrected  Pn/Sn  in  nine  frequency  bands  for  3  Novaya  Zemlya  nuclear 
explosions,  24  Steigen  earthquakes  and  the  950105  event,  from  ARCESS  recordings.  Distance 
corrections  applied  are  from  Sereno  (1990)  for  Fennoscandia.  Since  these  corrections  do  not 
necessarily  apply  to  the  Urals  event,  we  corrected  the  Novaya  Zemlya  explosions  and  Steigen 
earthquakes  relative  to  the  distance  at  which  the  950105  event  occurred  from  ARCESS.  Figure  21 
shows  that  corrected  Pn/Sn  values  for  the  950105  event  are  consistent  with  those  for  earthquakes 
above  3  Hz.  Pn/Sn  values  for  the  950105  event  are  significantly  attenuated  above  8  Hz. 
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Figure  19.  Uncorrected  Pn/Lg  measurements  in  eight  frequency  bands  for  the  950105  event  recorded  by 
ARU  and  OBN  and  earthquakes  and  nuclear  explosions  recorded  by  WMQ. 
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Figure  20.  Same  as  Figure  19,  but  for  uncorrected  Pn/Sn. 


Figure  21.  Distance-corrected  Pn/Sn  values  in  nine  frequency  bands  for  events  recorded  by  ARCESS. 


Figure  22  shows  distance-corrected  Pn/Lg  in  eight  frequency  bands  for  the  950105  event  seen  at 
OBN,  and  17  nuclear  explosions  and  23  earthquakes  recorded  by  WMQ.  Distance  corrections 
were  obtained  by  performing  a  least  squares  fit  to  Pn/Lg  as  a  function  of  distance  in  each 
frequency  band  using  the  earthquakes  recorded  by  WMQ.  The  reference  distance  in  this  case  was 
set  at  1275  km  to  correspond  to  the  distance  of  the  950105  event  from  OBN.  Figure  22  shows  that 
distance-corrected  Pn/Lg  values  for  the  950105  event  are  consistent  with  those  for  earthquakes 
above  2  Hz  and  considerably  different  than  those  for  the  nuclear  explosions.  Figure  23  shows  a 
similar  plot  of  distance -corrected  Pn/Sn  measurements.  Pn/Sn  values  in  all  bands  at  OBN  for  the 
950105  event  are  also  consistent  with  those  for  earthquakes  seen  at  WMQ. 

We  also  performed  comparisons  of  Pn/Lg  and  Pn/Sn  measurements  at  ARU  for  the  950105  event 
to  those  for  earthquakes  and  explosions  seen  at  WMQ.  In  this  case,  WMQ  events  were  distance- 
corrected  relative  to  the  370  km  distance  of  the  950105  event  from  ARU.  At  this  distance  range 
the  corrections  vary  rapidly  as  a  function  of  distance  and  there  were  only  a  few  WMQ  events  in 
this  ranse  with  which  to  estimate  the  corrections.  Thus,  until  further  data  is  obtained,  these 
corrections  are  suspect  for  distances  less  than  400  km  and  we  do  not  present  comparisons  here. 
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2.3.  Categorization  Results  for  the  950105  Event 

Here  we  categorize  the  950105  Urals  event  using  the  procedure  described  in  Section  1.  Since  the 
950105  event  was  onshore,  not  deeper  than  10  km  and  the  CMR  did  not  have  an  Ms 
measurement,  this  event  could  not  be  placed  in  Category  1.  Thus,  it  was  processed  further  using 
the  multivariate  population  analysis.  Since  ARCESS  is  the  only  common  station  or  array  for 
which  we  currently  have  data  for  this  event  and  other  reference  events,  we  applied  the  outlier 
method  to  the  950105  event  using  Pn/Sn  maximum  amplitude  measurements  in  the  3-5,  4-6,  5-7, 
and  6-8  Hz  bands,  and  using  the  24  Steigen  and  5  Spitsbergen  earthquakes  to  train  the  algorithm. 

Figure  24  shows  categorization  results  for  the  950105  event  (orid=273228).  Also  shown  are 
categorization  results  for  the  3  Novaya  Zemlya  nuclear  explosions  (orids  shown  in  black)  and  the 
29  Steigen  and  Spitsbergen  earthquakes  (orids  shown  in  white)  which  were  recorded  by 
ARCESS.  This  figure  shows  that  the  950105  event  falls  in  Category  4  which  contains  5  of  the  29 
earthquakes,  while  the  nuclear  explosions  fall  in  Category  10.  The  probability  that  an  earthquake 
would  fall  in  Category  4  or  lower,  based  on  the  outlier  likelihood  ratio  statistic,  is  50%,  while  the 
probability  that  an  earthquake  would  fall  in  Category  10  is  0.0001%.  Thus,  the  950105  event 
would  not  be  considered  an  outlier  of  the  earthquake  group  unless  we  are  also  willing  to  identify 
up  to  50%  of  the  earthquakes  as  outliers  of  the  same  group  to  which  they  actually  belong. 

2.4.  Conclusions  and  Future  Plans 

Preliminary  comparisons  show  that  the  950105  event  is  consistent  with  regional  earthquakes  seen 
at  station  WMQ  and  the  ARCESS  array,  based  on  high-frequency  Pn/Lg  and  Pn/Sn 
measurements.  The  results  further  show  that  this  event  is  considerably  different  than  previous 
nuclear  explosions  seen  at  either  WMQ  or  ARCESS,  particularly  after  applying  distance 
corrections  and  above  3  Hz,  at  which  frequencies  Pn/Lg  and  Pn/Sn  are  expected  to  discriminate 
explosions  from  earthquakes.  Results  presented  in  Section  2.3  show  that  this  event  falls  in 
Category  4,  consistent  with  regional  earthquakes  seen  at  ARCESS.  In  addition,  this  event  is  not 
identified  as  an  outlier  of  the  earthquake  group  at  WMQ,  while  it  is  identified  as  an  outlier  of  the 
explosion  groups  at  both  ARCESS  and  WMQ.  This  event  is  also  classified  as  an  earthquake  rather 
than  an  nuclear  explosion  using  our  classification  test  based  also  on  the  likelihood  ratio. 

These  results  are  provided  with  the  caveats  that  comparisons  of  ARU  and  OBN  measurements  to 
those  at  WMQ  may  be  subject  to  station  and  path  differences  and  the  propagation  distance  of  the 
950105  event  to  ARCESS  is  considerably  greater  than  those  for  the  ARCESS  reference  events. 
Figures  19  and  21  shows  strong  frequency  dependence  of  Pn/Sn  for  the  950105  event,  which  is 
likely  due  to  significant  attenuation  effects  over  a  far  regional  propagation  path  (1800  km). 
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Figure  24.  Categorization  results  for  the  950105  event,  29  earthquakes  in  the  Steigen  and  Spitsbergen 
regions,  and  3  nuclear  explosions  at  the  Novaya  Zemlya  test  site. 

To  circumvent  the  two  stated  caveats,  we  are  trying  to  obtain  seismic  data  for  16  PNEs  which 
occurred  in  the  same  region  as  the  950105  event  and  were  recorded  by  some  of  the  same  stations 
and  arrays.  We  also  plan  to  analyze  seismic  data  for  earthquakes  in  the  same  region  and  seen  at 
the  same  stations  (ARU  and  OBN)  and  the  KVAR  array.  Given  these  data,  we  will  be  able  to 
make  further  direct  comparisons. 

Further  work  may  also  needed  to  better  understand  why  high-frequency  regional  discriminants  for 
this  event  are  consistent  with  identification  as  an  earthquake,  while  teleseismic  Ms:mb  is 
reportedly  inconsistent  with  identification  as  an  earthquake  for  such  a  shallow  event.  A  simulation 
study  by  Walter  (1995)  suggests  that  the  950105  event  can  be  modeled  as  a  mine  collapse.  To 
study  this  further,  we  have  been  trying  to  obtain  an  Ms  measurement  for  the  950105  event.  We 
plan  to  report  on  the  results  and  an  approach  to  treat  such  anomalous  events  in  the  near  future. 
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3.  Assessment  of  Current  Event  Characterization  Capability  using 

GSETT-3  Alpha  Network  Data 


3.1.  Introduction 

Here  we  compile  various  statistics  regarding  1786  seismic  events  which  occurred  between  11 
January  1995  and  12  February  1995  and  were  detected  by  a  set  of  30  Alpha  stations  as  part  of 
GSETT-3  (Group  of  Scientific  Experts  Technical  Test-3).  Seismic  data  from  these  stations  were 
transmitted  to  the  CMR  (Center  for  Monitoring  Research)  where  the  records  were  analyzed.  The 
objective  of  this  study  is  to  assess  the  numbers  and  characteristics  of  events  that  can  be  expected 
to  be  observed  by  the  Alpha  stations  during  a  given  period,  including  the  availability  and  utility  of 
event  characterization  parameters  that  can  be  used  to  identify  them.  High-frequency  regional 
phase  amplitudes  and  Ms  were  computed  for  the  events  during  this  period,  as  well  as  location, 
depth  and  mb  parameters.  Specifically,  we  compute  the  number  or  distribution  of: 

•  Overall  and  regional  (A  <  20®)  events  detected  at  each  station; 

•  Events  versus  mb,  M^  and  Ms; 

•  Events  versus  area  of  90%  confidence  location  error  ellipse; 

•  Events  with  location  error  ellipses  entirely  offshore; 

•  Events  versus  hypocentral  depth  estimate 

•  Events  versus  the  number  of  depth  phases  detected; 

•  Events  with  Ms  and  mb  measurements; 

•  Regional  events  for  which  high-frequency  Pn/Sn,  Pn/Lg,  Pg/Sn  or  Pg/Lg  were  computed; 

•  Detecting  stations  per  event  for  which  regional  discriminants  are  available. 

We  examine  how  many  events  can  be  screened  with  high  confidence  as  due  to  natural  seismicity, 
based  on  teleseismic  measures  of  depth,  location  and  Ms:mb.  Of  the  remaining  events,  we 
examine  how  many  have  useful  regional  di,scriminants  such  that  they  could  potentially  be 
identified,  considering  primarily  high-frequency  regional  P/S  and  P/Lg  amplitude  ratios.  We  also 
assess  their  utility  with  respect  to  signal-to-noise  ratios.  Last,  we  evaluate  the  number  of 
remaining  ambiguous  events  lacking  adequate  event  characterization  data  to  identify  them,  based 
on  the  Alpha  network  which  existed  at  the  time  of  this  study. 

In  Section  3.2  we  describe  the  data  set  and  results  of  statistical  compilations.  In  Section  3.3  we 
discuss  the  utility  of  event  characterization  parameters.  In  Section  3.4  we  provide  some 
conclusions  and  recommendations  regarding  current  and  future  event  identification  performance. 
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3.2.  GSETT-3Data 


Figure  25  shows  locations  of  30  Alpha  stations,  as  well  as  epicentral  locations  of  1786  events 
which  occurred  between  11  January  1995  and  12  February  1995  and  were  reviewed  by  seismic 
analysts.  Origin,  association  and  event  characterization  data  for  these  events  were  retrieved  from 
the  origin,  origerr,  assoc  and  originamp  tables  in  the  IDCWDB  (International  Data  Center 
Working  Data  Base)  at  the  CMR.  Table  4  summarizes  the  names,  locations,  types  (single  station 
or  array)  and  dates  on  which  each  Alpha  station  began  operating  (but  not  necessarily  transmitting 
data  to  the  CMR).  Of  the  30  Alpha  stations,  11  are  arrays  and  the  remainder  are  single  3- 
component  stations. 


IDCWDB  EVENTS:  1995011-1995042 


Figure  25.  Locations  of  30  Alpha  stations  and  1786  seismic  events  recorded  by  them  between  11  January  and 
12  February  1995. 

3.2.1.  Events  Per  Alpha  Station 

Figure  26  shows  the  numbers  of  overall  and  regional  events  that  were  detected  by  each  Alpha 
station.  Of  the  1786  events  during  this  period,  there  were  a  total  of  634  events  within  regional 
distance  (A  <  20°)  of  at  least  one  of  the  30  Alpha  station. 
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Table  4.  Alpha  station  names,  operation  start  dates,  locations,  and  types. 


Station/Array 

On  Date 

Latitude 

Longitude 

Station/Array  Name 

Type 

-  ARCES 

1987273 

69.5349 

25.5058 

ARCESS  Array,  Norway 

ar 

ARMA 

1994175 

-30.4198 

151.6280 

Armidale,  Australia 

ss 

ASAR 

1987001 

-23.6664 

133.9044 

Alice  Springs  Array,  Australia 

ar 

BDFB 

1993189 

-15.6418 

-48.0148 

Brasilia,  Brazil 

ss 

BGCA 

1994256 

5.1761 

1 8.4242 

Bogoin,  Central  African  Republic 

ss 

BOSA 

1993056 

-28.6140 

25.2555 

Boshof,  South  Africa 

ss 

CPUP 

1994001 

-26.3306 

-57.3292 

Villa  Florida,  Paraguay 

ss 

DRLN 

1993348 

49.2560 

-57.5042 

Deer  Lake,  Canada 

ss 

FCC 

1967175 

58.7610 

-94.0870 

Fort  Churchill,  Canada 

ss 

FINES 

1993322 

61.4436 

26.0771 

FINESS  Array,  Finland 

ar 

GERES 

1990067 

48.8451 

13.7016 

GERESS  Array,  Germany 

ar 

LBNH 

1993226 

44.2401 

-71.9259 

Lisbon,  New  Hampshire 

ss 

LOR 

1963001 

47.2683 

3.8589 

Lormes,  France 

ss 

MAW 

1956181 

-67.6039 

62.8706 

Mawson,  Antarctica 

ss 

MBC 

1961291 

76.2420 

-119.3600 

Mould  Bay,  Canada 

ss 

MIAR 

1992267 

34.5457 

-93.5730 

Mount  Ida,  Arkansas 

ss 

MJAR 

1984092 

36.5427 

138.2070 

Matsushiro  Array,  Japan 

ar 

NORES 

1984277 

60.7353 

11.5414 

NORESS  Array,  Norway 

ar 

PDAR 

1991001 

42.7667 

-109.5579 

Pinedale  Array,  Wyoming 

ar 

PDY 

1993218 

59.6333 

112.7003 

Peleduy,  Russia 

ss 

PLCA 

1992300 

-40.7283 

-70.5500 

Paso  Flores,  Argentina 

ss 

SPITS 

1992311 

78.1777 

16.3700 

Spitsbergen  Array,  Norway 

ar 

STKA 

1991263 

-31.8769 

141.5952 

Stephens  Creek,  Australia 

ss 

TXAR 

1993244 

29.3338 

-103.6670 

TXAR  Array,  Texas 

ar 

VNDA 

1993362 

-77.5139 

161.8456 

Vanda,  Antarctica 

ss 

WALA 

1992136 

49.0586 

-113.9115 

Waterton  Lakes,  Canada 

ss 

WHY 

1993239 

60.6597 

-134.8806 

Whitehorse,  Canada 

ss 

WOOL 

1994157 

-31.0730 

121.6780 

Woolibar,  Australia 

ss 

WRA 

1965274 

-19.9426 

134.3394 

Warramunga  Array,  Australia 

ar 

YKA 

1963273 

62,4932 

-114.6053 

Yellowknife  Array,  Canada 

ar 
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Figure  26.  Number  of  overall  and  regional  events  per  Alpha  station. 


3.2.2.  Events  Versus  Location  and  Location  Uncertainty 

Figure  27  shows  the  distribution  of  events  versus  the  area  of  their  90%  confidence  location  error 
ellipses.  Roughly  half  of  the  events  have  90%  location  error  ellipses  with  areas  greater  than 
10,000  km^.  Figure  28  shows  a  scatter  plot  of  location  error  ellipse  area  versus  mb.  The  areas 
exhibit  relatively  little  dependence  on  mb.  It  is  expected  that  future  use  of  Beta  and  Gamma 
station  data,  as  well  as  data  from  additional  planned  Alpha  stations,  will  help  to  reduce  the  size  of 
the  location  error  ellipses. 

Comparisons  by  North  (1995)  of  event  locations  in  Canada,  based  on  Alpha  station  data,  to 
Canadian  NDC  locations  using  Beta  and  Gamma  station  data  indicate  that  a  significant  percentage 
(11/16)  of  the  90%  confidence  location  ellip.ses  do  not  contain  the  Canadian  NDC  locations, 
accounting  for  their  uncertainties  as  well.  This  may  indicate  that  there  are  significant  biases  in 
location  estimates  which  are  not  treated  in  the  location  uncertainty  analysis. 

Note  that  of  the  1786  events  during  this  period,  609  have  90%  confidence  location  error  ellipses 
entirely  offshore. 


3.2.3.  Events  Versus  Depth  and  Detected  Depth  Phases 

Figure  29  shows  the  distribution  of  events  versus  their  hypocentral  depth  estimates.  There  are 
1201  of  1786  events  (67%)  whose  depth  estimates  are  less  than  10  km  deep  or  undetermined. 
Figure  30  shows  the  distribution  of  events  versus  the  number  of  depth  phases  detected.  There  are 
only  59  events  for  which  at  least  one  depth  phase  was  detected.  Figure  31  shows  the  cumulative 
distribution  of  events,  with  or  without  at  least  one  detected  depth  pha.se,  versus  the  upper  bound  of 
the  95%  depth  confidence  interval.  There  are  292  of  1786  events  during  this  period  with  95% 
depth  confidence  intervals  deeper  than  10  km,  of  which  there  are  58  with  at  least  one  detected 
depth  phase.  Thus,  using  a  95%  confidence  interval  for  depth,  roughly  16%  of  the  events  can  be 
classified  as  deep  natural  events.  If  we  also  require  that  at  least  one  depth  phase  be  detected',  only 
3%  of  the  events  can  be  classified  as  deep  natural  events  with  high  confidence. 


DISTRIBUTION  OF  EVENTS  VS.  DEPTH 


1.  The  additional  requirement  that  at  least  one  depth  phase  be  detected  is  motivated  by  the  fact  that  potential  biases  in 
depth  and  location  estimates  may  exist  which  are  not  treated  when  simultaneously  computing  depth  confidence  intervals 
and  location  error  ellipses. 


DISTRIBUTION  OF  EVENTS  VS.  DEPTH  PHASES 


DEPTH  PHASES 

Figure  30.  Distribution  of  events  vs.  number  of  depth  phases  per  event. 
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igure  31.  Cumulative  distribution  of  events,  with  or  without  one  or  more  detected  depth  phases,  versus  the 
upper  bound  of  the  95%  confidence  interval  for  depth. 


3.2.4.  Events  Versus  Magnitude 


Figure  32  shows  distributions  of  the  events  as  functions  of  mb.  Mg  and  M^.  Note  that  of  the  1786 
events,  there  are  327,  1643  and  1388  events  for  which  mb.  Mg  and  Mj^,  respectively,  were  not 
measured.  Of  the  remaining  1459  events  for  which  mb  was  measured,  all  but  35  (roughly  2%) 
were  above  mb  3.  Comparisons  of  IDC  mb  values  to  those  from  the  QED  (Quick  Epicentre 
Determination)  of  the  U.S.G.S.  National  Earthquake  Information  Center  found  that  the  IDC  mb 
values  are  about  0.3  units  smaller  than  the  QED  magnitudes  (IDC  Performance  Report,  3 
February  1995).  This  suggests  that  roughly  98%  of  the  events  are  above  mb  3.3  and,  thus,  there 
are  relatively  few  mining  blasts  to  contend  with  for  now. 


MAGNITUDE 

Figure  32.  Distributions  of  events  versus  mb.  Mg  and  Ml- 

Figure  33  shows  a  scatter  plot  of  Mg  versus  mb  for  143  events  for  which  both  mb  and  Mg  were 
measured.  All  but  3  of  these  143  events  fall  above  the  line  defined  by  mb-Mg  =  1.2.  Figure  34 
shows  a  similar  plot  of  Mj^  versus  mb.  The  scatter  about  the  linear  trend  is  greater  than  one 
magnitude  unit.  This  has  serious  implications  regarding  the  usefulness  of  M^  as  an  accurate  or 
consistent  measure  of  source  size,  and  will  likely  require  regional  calibration. 
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3.2.5.  Regional  High-Frequency  Amplitude  Ratios 

Regional  phase  amplitudes  for  the  events  during  this  period  were  computed  by  automated 
software  at  the  CMR.  Absolute  maximum  amplitudes  were  measured  on  2-4,  4-6,  6-8  and  8-10 
Hz  rms  beams  within  predicted  group  velocity  or  time  windows  defined  in  Table  2  for  Pn,  Pg,  Sn 
and  Lg.  Similar  measurements  were  made  of  noise  amplitudes  in  predicted  pre-Pn,  pre-Pg,  pre-Sn 
and  pre-Lg  windows,  which  are  also  summarized  in  Table  5.  Note  that  the  eight  windows  for 
predicted  theoretical  arrivals  are  used  for  all  regions  and  stations. 


Table  5.  Description  of  regional  phase  amplitude  measurements. 


Phase 

Window  Starts 

Window  Ends 

Pn 

3  sec  before  Pn 

6.4  km/sec 

Pg 

6.3  km/sec 

6.0  km/sec 

Sn 

4.8  km/sec 

4.2  km/sec 

Lg 

3.6  km/sec 

3.0  km/sec 

Pre-Pn  noise 

8  sec  before  Pn 

3  sec  before  Pn 

Pre-Pg  noise 

6.4  km/sec 

6.3  km/sec 

Pre-Sn  noise 

4.9  km/sec 

4.8  km/sec 

Pre-Lg  noise 

3.7  km/sec 

3.6  km/sec 

Figure  35  shows  the  number  of  regional  events  associated  by  each  Alpha  station,  those  for  which 
Pn/Sn,  Pn/Lg,  Pg/Sn  and/or  Pg/Lg  were  computed,  and  the  subset  of  these  with  signal-to-noise 
ratios  (SNR)  greater  than  2  for  Pn  or  Pg  and  Sn  or  Lg.  (The  legend  associates  the  hatch  patterns  to 
these  three  cases.)  There  are  a  total  of  634  events  within  regional  distance  (A  <  20°)  of  at  least  one 
Alpha  station.  Except  for  MJAR,  regional  phase  amplitudes  were  computed  for  most  (487  of  634) 
regional  events.  For  ARCES,  FINES,  GERES,  LOR,  MJAR,  PDAR,  PLCA,  SPITS,  WALA, 
WHY  and  WOOL,  the  number  of  events  for  which  regional  phase  amplitudes  were  computed 
(given  in  the  originamp  table)  exceeds  the  number  of  associated  regional  events  in  the  assoc 
table.  (This  is  because  the  regional  phase  amplitudes  were  based  on  automated  time  windows,  not 
on  explicit  phase  picks.)  Only  249  of  the  487  events  had  SNR  greater  than  2  for  at  least  one 
regional  P  phase  and  Sn  or  Lg.  Thus,  less  than  40%  of  the  events  detected  within  regional 
distances  have  useful  high-frequency  regional  amplitude  ratios. 
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Figure  35.  Number  of  regional  events  per  Alpha  station. 


Figure  36  shows  distributions  of  events  versus  the  number  of  detecting  regional  stations  in  the 
assoc  table  per  event,  those  for  which  regional  discriminants  were  computed  and  those  for  which 
the  regional  discriminants  have  SNR  >  2.  There  were  1152  events  which  were  not  associated  by 
any  station  within  regional  distances.  All  but  5  of  the  634  events  in  the  assoc  table,  were 
associated  by  four  or  fewer  stations.  Regional  amplitudes  were  computed  for  some  events  at  as 
many  as  six  stations.  (Note  that  associations  for  some  of  these  regional  stations  were  missing  in 
the  assoc  table.)  However,  of  the  events  with  SNR  >  2  for  both  numerator  and  denominator  of  at 
least  one  regional  amplitude  ratio,  there  were  only  9  events  detected  by  three  or  more  stations. 
Thus,  events  which  cannot  be  identified  by  teleseismic  discriminants  will  likely  be  analyzed  using 
regional  discriminant  data  from  one  to  three,  or  possibly  four.  Alpha  stations,  at  least  until  the 
density  of  the  Alpha  network  changes.  Supplemental  regional  data  from  Beta  stations  may  also  be 
used  to  improve  identification  performance  for  regional  events. 
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DiSTRiBUTION  OF  EVENTS  VS.  STATIONS/EVEF 


Figure  36.  Distribution  of  regional  events  versus  the  number  of  stations  per  event. 


Figures  37-40  show  plots  of  Pn/Sn,  Pn/Lg,  Pg/Sn  and  Pg/Lg  amplitude  ratios,  respectively, 
computed  in  four  frequency  bands  (2-4,  4-6,  6-8  and  8-10  Hz),  for  each  Alpha  station.  The 
markers  are  coded  based  on  SNR:  circles  indicate  SNR  below  2  for  either  numerator  or 
denominator,  while  asterisks  indicate  SNR  greater  than  2  for  both  numerator  and  denominator. 
Note  that  the  majority  of  amplitude  ratios  are  plotted  as  circles,  indicating  that  most  amplitude 
ratios  do  not  satisfy  a  criteria  of  SNR  >  2  for  both  numerator  and  denominator.  Examination  of  the 
amplitude  ratios  also  indicates  that  there  are  a  very  wide  range  in  values  (e.g.,  from  10"^  to  10''"^  at 
station  WRA)  for  events  which  are  predominantly  earthquakes. 

Figures  41  and  42  show  similar  plots  of  Pn/Sn  and  Pn/Lg,  respectively,  in  the  same  four  frequency 
bands  versus  distance  for  the  regional  events  and  for  all  Alpha  stations.  Markers  are  defined  as  in 
Figures  37-40.  There  are  a  significant  number  of  amplitude  ratios  in  the  4-6,  6-8  and  8-10  Hz 
bands  which  are  identically  equal  to  1.0  due  to  treating  the  amplitudes  to  only  one  significant 
digit.  Most  events  fall  along  typical  attenuation  curves,  but  there  is  considerable  scatter. 


41 


Pn/Sn(2-4  Hz)  VERSUS  STATION  Pn/Sn(4-6  Hz)  VERSUS  STATION 
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Figure  37.  Pn/Sn  in  4  frequency  bands  versus  Alpha  station.  Circles  indicate  SNR  <  2  for  either  numerator 
or  denominator,  w^hile  asterisks  indicate  SNR  >  2  for  both  numerator  and  denominator. 
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Pn/Lg(2-4  Hz)  VERSUS  STATION  Pn/Lg(4-6  Hz)  VERSUS  STATION 
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Figure  38.  Same  as  Figure  37,  but  for  Pn/Lg. 
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Pg/Sn(2-4  Hz)  VERSUS  STATION  Pg/Sn(4-6  Hz)  VERSUS  STATION 
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Figure  39.  Same  as  Figure  37,  but  for  Pg/Sn. 
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Pg/Lg(2-4  Hz)  VERSUS  STATION  Pg/Lg(4-6  Hz)  VERSUS  STATION 
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Figure  40.  Same  as  Figure  37,  but  for  Pg/Lg. 
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Some  obvious  questions  arise  regarding  these  measurements.  First,  why  are  there  such  a  broad 
range  of  amplitude  ratios,  including  some  abnormally  high  values?  Second,  why  are  there  cases  in 
which  the  phase  amplitudes  are  greater  than  100  nm  but  the  SNR  is  less  than  2?  Third,  how  robust 
are  the  phase  windows  to  regional  variations  in  crustal  velocities?  That  is,  do  the  fixed  theoretical 
windows  contain  the  true  expected  arrivals  for  some,  most  or  all  of  the  regional  events?  Fourth, 
how  robust  are  these  windows  to  hypocentral  depth?  Some  of  these  questions  are  clearly  related. 
By  comparing  the  originamp  table  to  the  REB  for  12  events  with  very  high  amplitude  ratios 
(greater  than  100),  F.  Ryall  (1995)  found  the  following  peculiarities  associated  with  the  regional 
phase  amplitude  measurements. 

1.  Sn  times  in  the  originamp  table  are  inconsistent  with  arrival  times  in  the  REB  for  events  seen  at 
the  Australian  and  European  arrays. 

2.  Most  later  phases  for  the  12  events  have  very  small  amplitudes  (e.g.,  less  than  1.0)  and  are 
close  to  or  smaller  than  the  “noise”  amplitudes.  Almost  all  are  much  smaller  than  “Pn”  (e.g.,  by 
two  orders  of  magnitude).  For  example,  orid  286620  was  an  mb  5.2  event,  17.3  degrees  from 
WRA.  The  WRA  “Pn”  amplitude  for  this  event  is  6272.0,  while  the  “Pg”  amplitude,  which  is  only 
20  sec  after  “Pn,”  measures  1.9,  i.e.,  3300  times  smaller.  For  the  same  event,  the  “Sn”  amplitude 
is  0.2,  while  the  “pre-Pn  noise”  amplitude  is  0.1. 

2.  The  automated  program  measured  amplitudes  for  crustal  phases  (e.g.,  “Pg”)  for  events  which 
were  deep  and  for  which  the  REB  arrivals  were  P  and  S.  Examples  are  orids  277917  and  292133, 
for  which  the  depths  were  244  and  195  km,  respectively.  (Note  that  the  depth  confidence  intervals 
were  too  large,  however,  to  classify  these  events  as  deeper  than  10  km  at  a  95%  confidence  level.) 

3.  The  high-frequency  bandpass  (8-10  Hz)  gives  amplitudes  at  stations  ASAR  and  PDAR  that  are 
12  to  50  times  larger  than  the  other  bands,  for  all  12  events.  This  does  not  seem  reasonable  since 
ASAR  is  17-18  degrees  from  the  events.  Thus,  it  should  be  expected  that  high-frequency  content 
would  be  attenuated.  Also,  for  orid  299937,  PDAR  has  an  8-10  Hz  “Pn”  that  is  20  times  larger 
than  2-4  Hz  “Pn”  for  a  Gulf  of  California  event  (typically  low-frequency),  with  a  low-Q  path  of 
19  degrees  entirely  in  the  tectonically  active  western  North  America. 

4.  Amplitudes  are  calculated  for  phases  that  were  not  associated  (i.e.,  not  listed  in  the  REB  assoc 
table)  -  Pg  and  Lg  for  all  12  events,  as  well  as  Sn  for  orids  284853,  299937,  300444. 

5.  Amplitudes  are  given  for  one  station  that  was  not  listed  in  the  REB  and  could  not  have  recorded 
the  event.  For  orid  298017,  GERES  is  not  listed  in  the  REB,  but  has  very  large  “Pn”  and  “pre-Pn 
noise”  values  in  the  originamp  table.  This  event  was  in  Russia  and  was  probably  a  mine  blast. 
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Amplitudes  at  the  three  Scandinavian  arrays  are  quite  small  (less  than  10).  The  high  amplitudes 
listed  for  GERES  (423.0-1963.0)  were  possibly  for  a  noise  spike  or  another  event  near  the  array. 

Further  investigation  of  the  data  (including  the  seismic  waveforms)  by  D.  Jepsen  (1995)  provided 
the  following  conclusions.  First,  there  are  numerous  cases  for  which  large  noise  spikes  were  not 
masked  before  computing  the  high-frequency  amplitudes.  Figure  43  shows  an  example  of  seismic 
waveforms  for  an  event  recorded  by  the  Pinedale  array.  Large  amplitude  noise  spikes  were 
present  on  two  of  the  channels. 


Gulf  of  California  Event  -  GRID  299937 


Time  (Minutes) 


Figure  43.  Seismic  waveforms  for  a  Gulf  of  California  event  which  was  recorded  by  the  Pinedale  array. 
Large  amplitude  noise  spikes  were  present  on  two  of  the  channels. 
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Second,  the  theoretical  Sn  velocity  window  does  not  contain  the  actual  Sn  arrivals,  picked  by 
seismic  analysts,  for  the  majority  of  events  within  ten  degrees  from  an  Alpha  station  (Figure  44). 
The  window  (dashed  curves)  in  most  cases  was  prior  to  the  actual  Sn  arrivals,  leading  to 
unusually  small  Sn  values.  The  solid  line  depicts  the  IASPEI91  travel  time  curve  for  Sn  which  is 
more  consistent  with  the  analyst’s  picks.  Thus,  although  the  Sn  measurements  are  currently 
erroneous,  the  problem  appears  to  have  a  simple  solution. 


Figure  44.  Sn  travel  times  versus  distance  (stars)  based  on  analyst  picks  for  the  one-month  data  set.  Also 
plotted  are  the  IASPEI91  Sn  travel  time  curve  (solid  line)  and  the  theoretical  velocity  window 
(dashed  lines)  used  to  make  automated  Sn  measurements.  (Courtesy  of  D.  Jepsen.) 
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Third,  the  abnormally  large  8-10  Hz  amplitudes  for  some  events  is  due  to  applying  instrument 
response  corrections  in  a  frequency  band  near  the  Nyquist  frequency  for  some  stations  or  arrays 
where  the  instrument  response  falls  off  sharply.  That  is,  most  of  the  seismic  energy  in  this  band  is 
at  8  Hz,  while  the  correction  applied  is  based  on  the  instrument  response  at  9  Hz,  which  leads  to  a 
significant  over-correction.  Thus,  amplitudes  in  the  8-10  Hz  band,  at  stations  with  a  Nyquist 
frequency  of  10  Hz  or  less,  should  not  be  used  and,  possibly,  not  included  in  the  database  at  all. 

Although  there  are  several  significant  problems  with  the  high-frequency  phase  amplitude 
measurements,  virtually  all  of  the  problems  found  have  straightforward  solutions.  Current  work 
by  SAIC  includes  developing  software  to  better  mask  noise  spikes  and  modifying  the  predicted 
Sn  time  window  to  include  the  actual  arrivals.  Cross-checking  the  originamp  table  with  the  assoc 
table  in  the  REB  database  is  also  expected  to  alleviate  problems  with  interfering  events,  etc. 


3.3.  Utility  of  Event  Characterization  Parameters 

Here  we  discuss  the  utility  of  teleseismic  and  regional  event  characterization  parameters.  We 
focus  on  determining  how  many  events  can  be  characterized  as  due  to  natural  seismicity  and  on 
assessing  how  many  ambiguous  events  remain  due  to  a  lack  of  any  useful  discriminants.  Using 
depth,  Msimb,  location  (offshore  versus  onshore),  and  high-frequency  regional  P/S  and  P/Lg 
discriminants  to  characterize  the  events,  we  obtain  the  following  results  for  the  1786  events: 

•  292  events  have  95%  depth  confidence  intervals  deeper  than  10  km  and  are  thus  classified  as 
deep  natural  events. 

•  143  events  have  Ms  and  mb  measurements,  of  which  all  but  3  satisfy  mb-Ms  <  1.2.  Data  to 
compute  Ms:mb  confidence  intervals  were  unavailable.  (Only  6  of  the  143  events  were  also 
classified  as  deep.) 

•  454  remaining  events  (609  total)  have  90%  location  error  ellipses  entirely  offshore  but  there 
are  no  hydroacoustic  data  available  to  verify  that  these  events  were  not  offshore  explosions. 

•  324  remaining  events  (487  total)  have  high-frequency  regional  amplitude  ratios  available. 
Only  166  remaining  events  (249  total)  have  SNR  greater  than  2  for  both  numerator  and 
denominator  of  at  least  one  amplitude  ratio  for  at  least  one  regional  station. 

•  Overall,  883  of  1786  events  (49%)  can  potentially  be  identified,  based  on  depth,  location 
(plus  hydroacoustic  data  eventually),  and  Msimb.  Including  high-frequency  regional  phase 
amplitude  ratios  with  SNR  >  2,  a  total  of  1049  events  (59%)  can  potentially  be  identified'. 


I.  Note  that  peculiarities  regarding  the  regional  amplitude  measurements  must  be  resolved  before  a  definitive 
assessment  of  their  utility  for  event  characterization  can  be  obtained. 
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Thus,  at  best,  there  are  still  511  teleseismic  and  226  regional  events  which  cannot  be  identified 
based  on  available  information  considered.  That  is,  there  are  737  of  the  1786  events  (41%)  during 
the  period  1955011-1995042  which  cannot  be  classified  as  deep  or  offshore,  and  do  not  have 
M5:mb  or  high-frequency  regional  discriminants  with  adequate  SNR.  Figure  45  summarizes  these 
results.  The  bars  indicate  the  numbers  of:  (1)  total  events;  (2)  offshore  events;  (3)  deep  events;  (4) 
events  with  Mgimb;  (5)  events  with  regional  amplitude  ratios  (total  and  those  with  SNR>2);  (6) 
teleseismic  (T)  and  regional  (R)  events  with  one  or  more  of  the  considered  discriminants;  and  (7) 
teleseismic  and  regional  events  with  none  of  the  considered  discriminants. 

NUMBER  OF  EVENTS  PER  CATEGORY 

ALL  EVENTS 


OFFSHORE  EVENTS 

DEEP  EVENTS 

EVENTS  WITH  Ms:mb 

REGIONAL  AMPS 

DISCRIMINANTS 

NO  DISCRIMINANTS 

0  500  1000  1500  2000 

NUMBER  OF  EVENTS 

Figure  45.  Distribution  of  events  with  various  available  discriminants. 

Figure  46  shows  locations  of  the  1049  events  which  are  deep,  offshore,  have  M5:mb,  and/or  have 
at  least  one  regional  high-frequency  amplitude  ratio.  Events  for  which  regional  amplitudes  were 
computed  occurred  primarily  in  northern  Europe  and  Seandinavia,  in  or  near  Australia,  and  in 
Alaska  and  Canada.  Figure  47  shows  locations  (asterisks)  and  90%  error  ellipses  of  the  511 
teleseismic  events  which  cannot  be  classified  as  deep  or  offshore,  do  not  have  Msimb,  and  do  not 
have  any  regional  P/S  or  P/Lg  discriminants.  The  diamonds  indicate  the  locations  of  the  226 
regional  events  which  do  have  at  least  one  regional  P/S  or  P/Lg  discriminant,  but  with  SNR  less 


than  2  for  either  numerator  or  denominator  (or  both).  Alpha  stations  transmitting  data  to  the  CMR 
at  the  time  of  this  study  are  shown  in  black,  while  projected  Alpha  stations  are  shown  in  light 
gray.  Note  that  many  of  these  events  are  at  far  regional  to  teleseismic  distances  from  existing 
Alpha  stations,  but  would  be  well  within  regional  distances  of  future  Alpha  stations.  Thus, 
additional  planned  Alpha  stations  will  greatly  help  to  reduce  the  number  of  events  for  which  none 
of  the  considered  discriminants  are  available  or  regional  amplitude  ratios  have  insufficient  SNR. 


EVENTS  WITH  DISCRIMINANTS:  1995011-1995042 


Longitude  (degrees) 


Figure  46.  Locations  of  events  which  can  be  classified  as  deep,  offshore,  with  mb-Mg  <  1.2,  or  with  at  least 
one  high-frequency  regional  amplitude  ratio.  Markers  are  coded  by  category. 

There  are,  however,  a  significant  number  of  events  within  regional  distances  of  existing  Alpha 
stations  (e.g.,  from  MJAR  in  Japan)  for  which  no  regional  phases  were  associated,  even  after 
analyst  review.  For  events  which  occurred  between  1995001  to  1995027,  there  were  only  158  of 
444  events  within  regional  distances  for  which  Pn  or  Pg  and  Sn  or  Lg  were  among  the  associated 
phases.  Possible  explanations  for  this  include  (1)  regional  phase  attenuation  for  events  at  far 
regional  distances;  (2)  regional  phase  blockage  (e.g.,  for  events  southeast  of  GERES);  (3)  deep 
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regional  events  (e.g.,  near  MJAR)  which  do  not  produce  crustal  phases  but  whose  95%  depth 
confidence  intervals  were  not  deeper  than  10  km;  and  (4)  possible  phase  association  errors  by  the 
automated  system  which  were  not  corrected  after  analyst  review.  Examples  supporting  the  first 
three  explanations  have  been  found.  Study  of  relevant  waveforms  is  needed  to  determine  whether 
additional  regional  phases  could  be  identified  based  on  further  inspection  by  a  seismic  analyst. 


EVENTS  W/0  DISCRIMINANTS;  1995011-1995042 


Figure  47.  Locations  of  events  for  which  none  of  the  considered  teleseismic  and  regional  discriminants  are 
available. 

Last,  Figure  47  shows  epicentral  location  estimates  of  at  least  five  events  which  appear  to  be 
clearly  offshore,  e.g.,  two  in  the  north  Atlantic  Ocean,  one  in  the  south  Pacific  Ocean,  and  two  in 
the  Indian  Ocean  (one  south  and  one  west  of  Australia).  The  two  in  the  north  Atlantic  are  actually 
in  the  Azores  Island  region,  while  the  two  in  the  Indian  Ocean  and  the  one  in  the  south  Pacific 
have  90%  error  ellipses  which  overlap  either  Antarctica,  Madagascar  or  North  America.  Jepsen 
(1995)  found  that  several  of  the  events  with  extremely  large  error  ellipses  had  only  one  or  two 
arrivals,  on  which  these  location  estimates  were  based,  and  the  events  were  subsequently  omitted 
from  the  REB. 
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3.4.  Conclusions  and  Recommendations 

Based  on  the  analyses  described  above  for  events  which  occurred  between  11  January  1995  and 
12  February  1995  of  GSETT-3,  we  now  provide  some  conclusions  regarding  the  current 
capability  to  screen  seismic  events  based  on  available  information.  We  also  provide  some 
recommendations  to  make  necessary  improvements  and  to  study  unresolved  issues. 

First,  requiring  that  the  95%  confidence  interval  for  depth  be  deeper  than  10  km,  292  of  the  1786 
events  (16%)  are  classified  as  deep  natural  events.  If  we  also  require  that  at  least  one  depth  phase 
be  detected  (since  there  may  be  potential  biases  in  location  and  depth  estimates),  only  58  events 
are  classified  as  deep.  Thus,  it  is  important  to  resolve  the  issue  of  biases  in  location  and  depth 
estimates,  and  to  assess  whether  improvements  in  identifying  depth  phases  can  be  made. 

Second,  there  are  143  events  (8%)  with  Ms;mb  measurements.  This  low  percentage  is  due  to  the 
fact  that  Mg  measurements  are  currently  made  only  for  long-period  instruments  which  are  limited 
in  number.  Future  work  by  S-Cubed  includes  array  processing  for  broadband  instruments,  which 
should  lead  to  a  much  higher  percentage  of  events  with  Ms  measurements.  All  but  3  of  the  143 
events  satisfy  mb-Ms  <  1  -2;  however,  the  effect  of  magnitude  biases  has  yet  to  be  resolved  and 
parameters  to  compute  appropriate  confidence  intervals  are  still  needed. 

Third,  609  events  have  90%  location  error  ellipses  entirely  offshore,  but  there  were  no  hydro¬ 
acoustic  (HA)  data  available  to  verify  that  they  were  not  offshore  explosions.  Future  plans  by 
AREA  include  collection  of  HA  data  and  implementation  of  algorithms  to  compute  relevant  event 
characterization  parameters.  Incorporating  seismic  data  from  Beta  stations  could  lead  to  smaller 
error  ellipses,  thereby  improving  the  capability  to  classify  offshore  events  with  high  confidence. 

Fourth,  there  were  487  events  for  which  regional  phase  amplitudes  were  computed.  Of  these,  249 
have  at  least  one  Pn  or  Pg  phase  and  one  Sn  or  Lg  phase  with  SNR  greater  than  2.  We  found, 
however,  numerous  discrepancies  in  these  measurements,  specifically,  some  very  large  amplitude 
ratios  (e.g.,  greater  than  100)  for  events  which  are  presumed  earthquakes.  Many  of  the  anomalous 
values  are  due,  in  part,  to  large  unfiltered  noise  spikes  in  the  waveforms.  The  amplitude  ratios  also 
need  to  be  distance-corrected.  Theoretical  windows  used  in  computing  Sn  amplitudes  often  do  not 
coincide  with  the  actual  Sn  arrivals.  Work  to  resolve  these  discrepancies  is  under  way  and  appears 
promising.  Further  quality  control  is  needed,  however,  before  these  regional  discriminants  can  be 
used  with  confidence. 

Overall,  there  were  883  events  which  can  be  identified  by  teleseismic  measures  of  depth,  offshore 
location  and/or  Ms:mb.  An  additional  166  events  have  regional  discriminants  with  adequate  SNR, 


55 


although  further  work  is  needed  to  recompute  and  validate  these  measurements.  At  best  there  are 
still  511  teleseismic  and  226  regional  events  which  cannot  be  identified  based  on  available 
information  considered.  That  is,  there  are  737  of  the  1786  events  (41%)  during  the  period 
1955011-1995042  which  cannot  be  classified  as  deep  or  offshore,  and  do  not  have  Msimb  or 
high-frequency  regional  amplitude  ratios  with  adequate  SNR. 

These  results  indicate  that  better  global  coverage  is  needed,  which  should  be  provided  when 
additional  Alpha  and  Beta  stations  come  on-line.  ARPA  contractors  are  also  implementing  new 
discriminants  and  working  on  regional  calibration  issues  that  will  improve  magnitude,  location 
and  depth  estimation.  We  are  implementing  a  subsystem  for  event  characterization/screening 
which  will  be  used  to  objectively  assess  the  combined  utility  of  proposed  discriminants. 

Among  the  algorithms  to  be  integrated  is  the  multivariate  population  (i.e.,  outlier)  analysis 
described  in  Section  1.  In  this  approach,  it  is  assumed  that  events  above  mb  3  are  predominantly 
earthquakes  with  potentially  one  or  at  least  a  small  number  of  nuclear  explosions.  That  is,  we 
assume  that  there  will  be  very  few  chemical  mining  blast  above  mb  3.  Note  that  of  the  1459 
events  with  mb  measurements,  there  were  only  35  below  mb  3.  Accounting  for  the  bias  of  0.3 
units  in  the  IDC  mb  values  relative  to  QED  mb  values  implies  that  roughly  98%  of  the  events 
during  this  period  were  actually  above  mb  3.3.  This  suggests  that  the  majority  of  events  will  fall 
in  the  regime  where  application  of  the  outlier  analysis  is  valid. 

Since  events  may  be  observed  by  several  (e.g.,  2  to  5)  stations  within  regional  distances,  we  have 
also  been  addressing  the  problem  of  how  to  best  combine  regional  event  characterization  data 
from  multiple  stations  in  the  population  analysis.  We  describe  the  solution  to  this  problem  in  the 
following  section. 

Among  future  issues  that  need  to  be  addressed  are: 

•  The  adequacy  of  hydroacoustic  data  for  identifying  offshore  events; 

•  The  adequacy  of  infrasonic  data  for  identifying  atmospheric  or  shallow-buried  events; 

•  The  accuracy  of  location  and  depth  estimates  (i.e.,  are  there  significant  untreated  biases 
which  require  extensive  regional  calibration?); 

•  The  performance  of  our  multivariate  event  characterization  system  once  a  more  complete 
set  of  event  characterization  parameters  becomes  available. 
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4.  Optimizing  Multivariate  Network  Evidence  for  Outlier  Detection 

4.1.  Introduction 

Fisk  et  al.  (1993,  1994)  have  presented  and  applied  to  seismic  data  a  multivariate  approach,  based 
on  the  likelihood  ratio,  to  test  for  outliers.  In  general,  there  are  numerous  types  of  regional 
measurements  that  can  be  used  as  discriminants,  which  may  be  available  at  multiple  (e.g.,  2  to  5) 
Alpha  stations  and  it  is  not  clear  from  the  outset  how  to  best  combine  this  information  in  a  test  for 
outliers.  Here  we  address  a  fundamental  problem  of  how  to  utilize  multivariate  discriminant  data 
from  multiple  stations  in  order  to  optimize  the  power  of  the  outlier  test  for  fixed  false  alarm  rate. 

There  are  at  least  three  approaches  to  this  problem.  The  first  is  to  simply  insert  all  available 
features  that  are  thought  to  discriminate  into  the  likelihood  ratio.  The  second  is  to  perform  an 
optimal  weighting  of  some  or  all  of  the  discriminants  first  and  then  insert  the  weighted 
combinations  into  the  likelihood  ratio.  The  third  is  to  perform  separate  tests,  based  on  the 
likelihood  ratio  and  subsets  of  discriminants,  calling  an  event  an  outlier  if  it  is  found  to  be  an 
outlier  by  any  of  the  individual  tests.  The  significance  levels  of  the  individual  tests,  in  this  case, 
must  be  appropriately  modified  to  maintain  the  overall  false  alarm  rate  of  the  combined  results. 

In  Section  4.2,  analytic  expressions  are  derived  for  the  power  of  each  of  these  tests.  In  Section 
4.3,  a  parametric  study  is  presented  in  which  the  power  of  the  three  tests  are  compared  in  order  to 
determine  conditions  for  which  a  particular  test  is  favored  over  the  others.  In  Section  4.4,  some 
conclusions  and  recommendations  are  provided  for  operational  application  to  seismic  monitoring. 

4.2.  Technical  Approach 

Suppose  there  are  two  populations  of  events  7t/  and  712  earthquakes  and  explosions)  and  that 
a /7-dimensional  vector  of  discriminants,  A"  =  ...,  A  j ,  is  used  to  characterize  the  event 

type  for  each  event.  Given  n  previous  observations  from  TCj  only,  x-  for  /  =  1,  ...,  n,  allocation 
of  a  new  observation,  x,  as  an  outlier  or  not  is  accomplished  by  testing  the  hypothesis 

//q!  A'j,7:2,  ...,X^j,X  G  71,  versus  //, :  TC,,  ...,  X  ^  G  7t,,  X  g  7C,  .  (EQ1) 

The  likelihood  ratio  can  be  used  to  test  whether  the  new  event  is  an  outlier  of  the  first  population 
or  not.  Under  the  assumption*  that  A-  xj  if  A  G  Tt^  it  can  be  shown  (e.g.,  Anderson, 

1984,  Chapter  5)  that  the  likelihood  ratio  statistic  to  test  a  new  observation,  x,  is  given  by 


1 .  Note  that  if  the  discriminant  data  do  not  have  a  multivariate  normal  distribution,  the  Box-Cox  transformation  (Box 
and  Cox,  1964)  can  be  used  to  transform  the  data  to  approximate  normality. 
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are  maximum  likelihood  estimates  of  )J,  and  E ,  respectively.  It  can  further  be  shown  that 
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where  is  the  noncentral  F-distribution  with  p  and  n-p  degrees  of  freedom  and 

noncentrality  parameter  5  =  'Ap,  where  Ap  =  p^^^  -p^'^  if  xg  7C. .  Note  that  if 

the  new  event  is  from  Kj,  the  noncentrality  parameter  is  zero,  yielding  a  central  F-distribution. 


Small  values  of  X  provide  evidence  against  Hq.  Thus,  the  outlier  test  is:  reject  Hq  if  X<  X^,^  where 
Xq^  is  the  critical  value  such  that  the  test  has  a  significance  level.  That  is,  Xq^  or,  equivalently, 
is  set  such  that  |i7J  =  pf F  1=  a,  using  the  central  F- 

distribution.  The  power  of  the  test  is  the  probability  of  rightfully  rejecting  Hq  when  x  G  7t| ,  and  is 
giver,  by  ,  using  the  noncentral  F-distribution.  Note 

that  if  the  sample  size  n  is  sufficiently  large,  the  noncentral  F-distribution  becomes  the  noncentral 

'y  ^  _ j 

X”-distribution  with  p  degrees  of  freedom  and  noncentrality  parameter  5  =  Ap'Z  Ap . 


Now  consider  three  separate  hypothesis  tests  which  differ  by  how  the  p-dimensional  discriminant 
vector  is  used  as  input  into  the  likelihood  ratio.  We  assume  for  now  that  the  sample  size  is 
sufficiently  large  so  that  the  %^-distribution  may  be  used.  The  three  tests  are  defined  as  follows: 


Test  1:  Full  Vector  Case:  Insert  the  full  p-dimensional  vector  for  the  training  data  and  new  event 
into  the  expression  for  X.  Using  the  central  X"-distribution  with  p  degrees  of  freedom,  set  critical 
value  so  that  the  test  has  a  significance  level.  Compute  the  power  using  a  noncentral  %~- 
distribution  with  p  degrees  of  freedom  and  noncentrality  parameter  given  by  Ap'Z  '  Ap . 


Test  2:  Minimum  Variance  Weighting  Case:  Compute  the  linear  combination  of  the  p 
discriminants,  z  =  a'x,  where  a'  =  (c/j,  a,,  ...,  minimizes  the  variance  of  z,  subject  to  the 
constraint  that  a'p  =  1  for  (3  =  ( 1,  1,  ...,  1)  .  It  may  be  shown  that  the  mean  and  variance  of  z 

7  7 

are  given  by  p.  =  ^t'p  and  a"  =  a  Ta .  The  vector  a  which  minimizes  al  is  given  by 
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a  =  IL  '(3/p'Z  'p.  Hence,  a"  =  1/p'Z  'p.  (Note  that  if  the  correlations  are  zero,  this 
procedure  is  equivalent  to  inverse  variance  weighting  of  the  p  discriminants.)  Insert  values  of  the 
scalar  z  for  the  training  data  and  new  event  into  the  expression  for  A,.  Using  the  central  x~- 
distribution  with  1  degree  of  freedom,  set  critical  value  so  that  test  has  a  significance  level. 
Compute  the  power  using  a  noncentral  x“-squared  distribution  with  1  degree  of  freedom  and 

9  9  I  _1  _] 

noncentrality  parameter  given  by  Ap.“/a”  =  1  P'S  A|J.  j  /P'S  p. 

We  also  examine  two  variations  of  Test  2.  In  the  first,  we  use  inverse  variance  weighting  where 
a  =  S  *P/P'S  '  P  as  above,  but  with  the  correlations  set  to  zero.  Alternatively,  we  consider  the 
minimum  variance  combination  with  the  further  restriction  that  all  components  of  a  are  positive. 
These  two  variations  of  Test  2  are  referred  to  as  Tests  4  and  5,  respectively.  As  we  show  below, 
neither  of  these  alternatives  provide  an  improvement  in  power  under  the  assumptions  considered. 

Test  3:  “At  Least  One”  Case:  Insert  each  component  of  the  p-dimensional  discriminant  vector  into 
separate  expressions  for  A.  Using  the  central  x"-distribution  with  1  degree  of  freedom,  set  the 
critical  values  so  that  each  test  has  dip  significance  level.  (An  event  is  considered  an  outlier  of  7i| 
if  it  is  found  to  be  an  outlier  by  any  of  the  individual  tests.  Note  that  setting  the  significance  level 
of  each  test  to  alp  yields  an  overall  significance  level  less  than  or  equal  to  a.)  Compute  the 
overall  power  of  the  test  by  simulating  new  observations  from  7^2  and  determining  the  percentage 
of  events  for  which  either  discriminant  value  violates  the  respective  threshold. 

4.3.  Parametric  Power  Comparison 

We  performed  a  parametric  study  to  assess  the  conditions  for  which  a  particular  test  is  preferred 
over  the  other  two,  based  on  its  relative  power.  Assume  for  now  that  p  =  2,  i.e.,  that  there  are  two 
scalar  components  of  the  discriminant  vector.  For  example,  the  two  components  could  be  Pn/Lg 
and  Pn/Sn  or  Pn/Lg  in  two  different  frequency  bands  at  a  single  station,  or  Pn/Lg  in  the  same  band 
at  two  different  stations.  (In  Section  4.4,  we  will  discuss  how  these  results  relate  to  the  general 
case  of  p  discriminants.)  In  the  following,  we  set  the  significance  level  at  a  =  0.01.  Also  setting 
a,  =  1  without  loss  of  generality,  the  free  parameters  are  p,  Apj,  A|i,  . 

Figure  48  shows  a  comparison  of  tabulated  power  of  each  of  the  three  tests  for  =  1  and 
various  values  of  the  correlation  coefficient  from  -0.6  to  -1-0.9.  Each  plot  shows  values  of  the 
power  for  fixed  p  and  various  values  of  Ap  j ,  Ap,  from  0  to  4.  The  numbers  for  each  parametric 
case  are  ordered  by  test  number,  i.e.,  upper,  middle  and  lower  values  are  for  Tests  1,  2  and  3, 
respectively.  Figure  49  is  similar  to  Figure  48,  except  that  0^  =  2.  Examination  of  the  tables  in 
Figures  48  and  49  indicates  that  there  is  no  single  test  which  provides  the  greatest  power  under  all 
parametric  conditions.  This  is  further  illustrated  in  Figures  50  and  51  which  show  power  curves 
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for  the  three  tests  versus  correlation  for  various  values  of  A|l,  and  Afi, .  The  solid,  dashed  and 
dotted  curves  represent  the  powers  of  Tests  1,  2  and  3,  respectively.  Note  that  the  power  of  Test  3 
typically  decreases  as  the  correlation  increases,  although  this  test  is  the  least  sensitive  to 
correlation  as  might  be  expected.  The  power  of  Test  2  decreases  monotonically  with  increasing 
correlation  if  the  variances  are  equal  since  Alt,  is  independent  of  the  correlation  for  this  case, 
while  ar  increases  monotonically  with  increasing  correlation.  The  power  of  Test  1  also  decreases 
monotonically  with  increasing  correlation  until  the  correlation  becomes  relatively  high,  provided 
A|i|  and  A|i,  are  different.  This  occurs  because  Test  1  detects  outliers  with  inconsistencies  in  the 
correlation  structure. 

Note  that  Test  2  has  the  greatest  power  if  the  correlation  is  sufficiently  small,  depending  on  A)i| 
and  A|l, .  Test  3  can  have  the  greatest  power  if  the  correlation  is  in  a  particular  range,  which  also 
depends  on  A|i,  and  Afi., .  This  observation  makes  it  tempting  to  establish  a  rule  whereby  we  use 
a  particular  test  if  the  correlation  is  in  a  particular  range.  Unfortunately,  the  danger  associated 
with  doing  this  lies  in  the  fact  that  we  will  not  know  Ap.,  and  Ajx,  in  most  situations.  Thus,  it  is 
not  clear  how  to  establish  a  criterion  on  the  correlation  which  is  valid  for  all  possible  values  of 
Ap.,  and  Ap., .  For  example,  Test  2  can  yield  extremely  poor  power  if  Ap,  and  Apj  are 
considerably  different. 

If  we  are,  however,  confident  that  Ap,  and  Ap.,  are  not  dramatically  different,  such  a  rule  can  be 
established  of  the  form: 

•  If  G]  and  G2  are  significantly  different,  use  Test  2. 

•  If  G]  and  G2  are  similar  and  p  <  0,  use  Test  2. 

•  If  G]  and  G2  are  similar  and  p  >  0,  use  Test  1  (or  possibly  Test  3  if  0  <  p  <  0.6). 

Note  that  while  there  are  ranges  of  the  parameters  for  which  Test  2  or  Test  3  have  the  greatest 

power,  Test  1  generally  has  comparable  or  greater  power  than  the  other  two  tests  over  all 
parameters  and  there  are  no  situations  in  which  using  Test  1  yields  dramatically  poor  results.  To 
illustrate  this  further.  Figures  52  and  53  show  the  difference  in  the  powers  of  Tests  2  and  1  and 
Tests  3  and  1,  respectively,  versus  correlation  for  various  values  of  Ap,  and  Ap.,  between  0  and 
4.  Figure  52  shows  that  while  there  are  cases  for  which  Test  2  can  provide  roughly  a  10%  increase 
in  power  over  Test  1,  there  are  many  cases  for  which  Test  2  can  provide  dramatically  poorer 
power,  particularly  if  G]  =  G2.  Figure  53  shows  that  while  there  are  also  cases  for  which  Test  3  can 
provide  roughly  a  5%  to  10%  increase  in  power  over  Test  1,  there  are  many  cases  for  which  Test  2 
can  provide  dramatically  poorer  power  than  Test  1  (e.g.,  by  as  much  as  90%),  depending 
primarily  on  the  correlation  and  the  values  of  Ap,,  and  Ap,  . 
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Figure  48.  Power  comparison  of  the  three  tests  for  Gj  =  07  =  F 
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Figure  49.  Power  comparison  of  the  three  tests  for  Oi  =  1  and  02  =  2. 
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Figure  51.  Same  as  Figure  50,  except  that  <72  =  2. 
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Although  we  will  eventually  have  a  considerable  amount  of  information  from  earthquake  data 
regarding  the  covariance  matrix,  we  will  not  know  Ap.j  and  in  most  situations.  Hence,  we 
will  not  know  which  of  the  curves  in  Figures  52  and  53  apply.  Thus,  the  most  robust  procedure  is 
to  use  Test  1  in  which  the  full  vector  of  discriminants  is  inserted  into  the  likelihood  ratio. 

To  illustrate  how  the  powers  of  the  three  tests  compare  for  a  real  situation,  we  now  present  results 
based  on  Pn/Lg  measurements  in  the  6-8  Hz  band  at  stations  KNB  and  MNV  for  35  earthquakes 
and  69  nuclear  explosions  on  the  Nevada  Test  Site.  In  this  case,  p  =  0.228,  6|  =  0.485, 
^2  =  0.416  =  0.856a,,  A|l,  =  1.104  =  2.276a,,  and  Ap.,  =  1.662  =  3.427a,.  Treating 
these  estimates  as  approximately  equal  to  the  corresponding  true  parameter  values  and  setting  the 
significance  level  a  =  0.01,  the  powers  of  Tests  1,  2  and  3  are  0.922,  0.954  and  0.926, 
respectively.  All  three  tests  have  comparable  power  in  this  case.  However,  since  Ap,  and  Ap2 
are  not  dramatically  different  and  the  correlation  is  relatively  small,  it  should  be  expected  that 
Test  2  would  have  the  greatest  power. 

For  comparison,  the  power  of  the  tests  at  0.01  significance  level  using  either  one  of  the  stations  is 
0.363  for  KNB  and  0.938  for  MNV.  Thus,  combining  data  from  the  two  stations  has  comparable 
power  to  using  only  the  best  station  and  does  not  require  that  we  know  which  station  provides 
greater  power  which,  in  practice,  we  will  not  know  in  most  situations. 

Thus,  for  cases  in  which  we  know  Ap,  and  Ap2 ,  criteria  may  be  established  to  select  a  particular 
outlier  test  with  the  highest  power.  Note,  however,  that  since  we  have  this  information,  we  could 
perform  a  statistical  classification  test,  which  has  greater  power  than  any  of  these  outlier  tests, 
using  fixed  criteria  or  training  data  for  both  event  types. 

We  now  consider  two  final  cases  which  are  variations  of  Test  2.  In  the  first,  we  use  inverse 
variance  weighting,  rather  than  the  minimum  variance  combination.  In  the  second,  we  use  the 
minimum  variance  combination  with  the  further  restriction  that  all  components  of  a  are  positive. 
We  compare  the  power  of  these  tests  (Tests  4  and  5,  respectively)  to  the  power  of  Test  2.  Note  that 
if  a,  =  <52,  all  three  of  these  tests  are  equivalent.  Hence,  we  set  a,  =  1  and  Oo  -  2. 

Figure  54  shows  power  curves  for  Tests  2,  4  and  5  versus  correlation  for  various  values  of  Ap, 
and  Ap.,  from  0  to  3.  Here  the  solid,  dashed  and  dotted  curves  represent  the  powers  of  Tests  2,  4 
and  5,  respectively.  Figure  54  illustrates  that  Test  2  generally  has  the  greatest  power  of  these  three 
tests.  Test  2  provides  considerably  greater  power  for  cases  in  which  the  correlation  is  particularly 
high  and  Ap,  is  greater  than  Ap.,.  Note  that  if  the  correlation  is  greater  than  a,/a2  =  0.5,  the 
power  of  Test  5  is  independent  of  the  correlation  since  only  the  first  discriminant  (with  the 
smallest  variance)  is  being  used  in  these  cases. 
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To  further  illustrate  these  points.  Figure  55  shows  the  difference  in  the  powers  of  Tests  4  and  2 
versus  correlation  for  various  values  of  AfJ.|  and  A|l,  between  0  and  4.  While  there  are  cases  for 
which  Test  4  provides  a  slight  improvement  in  power  over  Test  2,  there  are  many  cases  for  which 
Test  4  provides  dramatically  poorer  power,  particularly  if  p  is  negative  and  large  and  Ap,  is 
greater  than  Apj  or  if  p  is  positive  and  large  and  Apj  is  greater  than  A|a,  .  Thus,  Test  2  is  more 
robust  than  Test  4  under  the  assumptions  considered. 


P4-P2  VS.  p:  (72  =  2.0 


Figure  55.  Difference  in  the  power  of  Test  4  and  Test  2  versus  correlation  for  various  values  of  Apj  and 
AfX.,  from  0  to  4. 


Figure  56  shows  a  similar  plot  comparing  the  powers  of  Tests  5  and  2  versus  correlation  for 
various  values  of  A|i]  and  A|J.,  between  0  and  4.  Note  that  these  two  tests  provide  equivalent 
power  unless  the  correlation  is  greater  than  <3^102  -  0.5.  While  there  are  cases  for  which  Test  5 
provides  greater  power  than  Test  2,  there  are  many  cases  for  which  Test  5  provides  dramatically 
poorer  power,  particularly  if  p  and  Ap,  (corresponding  to  the  discriminant  with  the  smallest 
variance)  are  large.  Thus,  Test  2  is  also  more  robust  than  Test  5  under  the  assumptions  considered. 
Hence,  consideration  of  these  two  variations  of  Test  2  do  no  impact  our  previous  conclusion  that 
Test  1  is  the  most  robust  outlier  test  to  use  under  the  widest  range  of  reasonable  assumptions. 
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P5-P2  VS.  p:  (J2-2.0 


P 

Figure  56.  Difference  in  the  power  of  Test  5  and  Test  2  versus  correlation  for  various  values  of  Ajl|  and  A[i2 
from  0  to  4, 


4.4.  Conclusions  and  Recommendations 

In  Section  4.2,  we  presented  three  outlier  tests,  all  based  on  the  likelihood  ratio,  which  utilize 
input  data  in  different  forms.  Based  on  the  parametric  study  presented  in  Section  4.3,  it  is  apparent 
that  there  is  no  single  test  which  provides  the  greatest  power  under  all  conditions.  In  practical 
monitoring  situations,  one  of  three  cases  will  apply:  (1)  we  will  know  (or  at  least  be  able  to 
estimate)  all  of  the  parameters  if  we  have  sufficient  training  data  for  both  earthquakes  and  nuclear 
explosions;  (2)  we  will  know  (or  at  least  be  able  to  estimate)  the  covariance  matrix  if  we  have 
sufficient  training  data  for  one  event  type  (e.g.,  earthquakes),  and  we  may  be  able  to  infer  a  range 
of  reasonable  values  for  A|i|  and  A|i,  from  previous  experience  in  other  similar  regions;  (3)  we 
will  know  (or  at  least  be  able  to  estimate)  the  covariance  matrix,  ^t  we  will  not  have  sufficient 
information  regarding  A|i  j  and  A|i., . 

Under  the  first  scenario,  we  could  select  the  outlier  test  with  the  greatest  power.  Alternatively,  we 
could  perform  a  statistical  classification  test,  with  even  greater  power,  using  training  data  for  both 
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event  types.  In  this  case,  it  can  be  shown  that  if  the  discriminant  vector  has  a  multivariate  normal 
distribution  and  the  covariance  matrices  for  the  earthquakes  and  explosions  are  equal,  the 
classification  test  with  the  greatest  power  is  based  on  the  linear  discriminant  function  (e.g., 
Anderson,  1984,  p.  205)  using  the  full  vector  (i.e.,  with  no  prior  weighting  of  the  discriminants). 

Under  the  second  scenario,  there  may  be  sufficient  information  to  select  the  best  outlier  test  if 
Ap.,  and  Ap,  can  be  inferred  reasonably  well  and  if  Ap,  is  not  dramatically  different  than  Ap2 . 

Under  the  third  scenario,  which  will  also  be  the  most  common,  we  will  not  know  Ap,,  Apj  and, 
hence,  we  should  use  the  outlier  hypothesis  test  in  the  form  of  Test  1,  using  the  full  discriminant 
vector  as  input  into  the  outlier  likelihood  ratio.  The  practical  difficulty  with  using  Test  1  is  that 
unless  there  is  a  lot  of  data,  it  may  be  difficult  to  get  a  good  estimate  of  the  covariance  matrix  for 
cases  in  which  we  have  a  large  number  of  discriminants.  This  limitation  should  become  irrelevant 
since,  after  a  sufficient  period  of  time,  a  large  earthquake  database  will  be  established  with  which 
to  estimate  the  covariance  matrix  for  all  discriminants  at  all  combinations  of  stations  that  will 
observe  any  regional  event. 

As  a  final  note,  there  will  generally  be  far  more  than  two  discriminants  u,sed  in  an  operational 
setting.  Although  the  same  three  monitoring  scenarios  still  apply,  the  parameter  space  for  which 
one  outlier  test  is  chosen  over  the  others  becomes  far  more  complicated  under  the  second  scenario 
and  may  require  substantial  a  priori  knowledge  regarding  discriminant  means  for  explosions. 
Thus,  we  recommend  using  Outlier  Test  1  for  all  cases  in  which  we  do  not  have  regional  training 
data  for  nuclear  explosions  and  we  recommend  using  a  multivariate  classification  test  for  cases  in 
which  we  do. 
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